現今製造業處於競爭激烈環境下，面臨必須有效掌控訂單生產進度，快速反應客戶需求，以提昇企業競爭優勢，所以前人的研究中提出現場生產監控系統（Shop Floor Control System, SFCS）可以用來解決上述之問題。傳統以類神經網路或決策樹等機器學習方法，在SFCS中設計控制器（排程控制器），需經由有限的訓練樣本中歸納排程知識。其主要缺點為訓練樣本的類別（排程決策規則）必須事先已知。然而此一取得訓練樣本的程序在設計多控制器於SFCS時會變成無法忍受的耗時工作，其原因為在下一個排程階段需決定多個排程法則。除此之外，以類神經網路或決策樹等機器學習方法所歸納排程知識為區域搜尋求解的方法，對全域目標函數（即整體生產績效指標）的達成仍存有問題。亦即每一個排程控制器之最佳排程法則可決定，但組合後之排程法則無法同時滿足全域目標函數。然而目前相關研究文獻甚少專注於如何進行多個控制器協調以達到整體生產績效指標，因此本研究設計一加強式學習為基之多控制器，以達到上述目標。
本研究提出之多控制器內容包含如下五個模組：模擬為基訓練樣本產生機制、資料正規化機制、訓練樣本以兩階段自我組織映射類神經網路（two-level SOM）之群集分析、以SOM為基之即時排程類別選擇機制及最重要的單元：加強式學習（Reinforcement Learning）為基之多控制器。本研究所提出以加強式學習為基之多控制器結合SOM及加強式學習的方法，可根據不同的系統屬性、目前的排程法則及績效指標，在一特定的排程類別中選擇適當排程（多個一組）規則。在加強式學習為基之多控制器模組中，先經由兩階段SOM分群方法以建立不同的排程類別，然後再使用本研究所提加強式學習演算法於特定的排程類別中選擇適當排程規則。經由本研究所提出之多控制器的方法，相對於使用隨機排程機制的方法及單一啟發式派工法則，應用在半導體製造系統與彈性製造系統，長期而言期望其方法可以有效改善其生產績效。另外，因為半導體製造系統擁有龐大且複雜的作業環境，不能只是單一控制區的排程控制而需要多個控制區的排程整合，所以無需考慮到單控制器的方法。但是在彈性製造系統的作業環境中，可以允許單一控制區的排程控制，所以本研究在彈性製造系統的實驗對照組中，多增加一個與彈性製造系統單控制器的實驗結果比較（這裡單控制器採用的方法為GA/DT與GA/SVM兩種對照組）。實驗結果在選定的績效標準下做評估比較，這裡本研究選定平均週期時間以及產出量來作為實驗評估的績效指標。

In the current competitive environment, in order to enhance enterprise competitive advantage, manufacturing needs to control the order process of production and fast response customer demand. But in the conventional machine learning approach, such as artificial neural network （ANN）, decision tree（DT）develop an controller （i.e. scheduling controller）in shop floor control system（SFCS）to induce scheduling knowledge from a limited set of training examples. It has the main disadvantage that the classes (scheduling decision rules) to which training data are assigned must be given. However, this process becomes an intolerable time-consuming task to develop multi-controller for SFCS because of multi-decision scheduling rules for next scheduling period horizon must be determined. In addition, the local approach, whether it is based on ANN or DT learning, has the problem of global objective function （i.e. overall production performance）. That is, although the best decision rule can be selected for each scheduling controller, the combination of the selected decision rules would not simultaneously satisfy the global objective function. However, there is still very little research focusing on harmonized multi-controller for SFCS to achieve global performance criterion. To resolve above problems, this study provides a real time scheduling decision rule knowledge base（KB）to support production, multi-decision rules for next scheduling period horizon must be determined. Hence, in this study develops multi-controller to achieve these goals.
The proposed multi-controller SFCS comprise five components: simulation-based training example generation mechanism, data normalization mechanism, training example clustering by two-level self-organizing map（SOM）approach, SOM-based real time decision rules class selection mechanism and the most important component: reinforcement learning（RL）-based multi-controller mechanism. In this study, the RL-based multi-controller combines SOM clustering and RL approach to select proper of decision rules in specific scheduling decision rules class based on various system statuses, current scheduling decision rules and performance criterion. Here, a number of scheduling decision rules classes are generated by two-level SOM clustering approach. Then, select proper of decision rules in specific decision rules class by proposed RL algorithm for multi-controller. Applying this approach in semiconductor FAB and flexible manufacturing system（FMS）, we expect the experiments will be demonstrate that the proposed multi-controller approach will lead to improve production performance compared to random scheduling mechanism and the heuristic individual dispatching rule based on various performance criteria.

摘要．．．．．．．．．．．．．．．．．．．．．．．．．．．．．Ⅰ
ABSTRACT．．．．．．．．．．．．．．．．．．．．．．．．．．．Ⅲ
目錄．．．．．．．．．．．．．．．．．．．．．．．．．．．．．Ⅴ
表錄．．．．．．．．．．．．．．．．．．．．．．．．．．．．．Ⅶ
圖錄．．．．．．．．．．．．．．．．．．．．．．．．．．．．．Χ
一、緒論．．．．．．．．．．．．．．．．．．．．．．．．．．．1
1.1 研究背景．．．．．．．．．．．．．．．．．．．．．．．．．1
1.2 研究動機．．．．．．．．．．．．．．．．．．．．．．．．．4
1.3 研究目的．．．．．．．．．．．．．．．．．．．．．．．．．4
1.4 研究流程．．．．．．．．．．．．．．．．．．．．．．．．．5
二、文獻探討．．．．．．．．．．．．．．．．．．．．．．．．．6
2.1 現場生產監控系統．．．．．．．．．．．．．．．．．．．．6
2.2 半導體製造系統．．．．．．．．．．．．．．．．．．．．．6
2.3 彈性製造系統．．．．．．．．．．．．．．．．．．．．．．9
2.4 即時排程法則選擇機制之相關研究．．．．．．．．．．．．．11
2.5 Las Vegas Filter屬性篩選演算法．．．．．．．．．．．．．12
2.6 代理人系統．．．．．．．．．．．．．．．．．．．．．．．15
2.7 加強式學習．．．．．．．．．．．．．．．．．．．．．．．16
2.8 自我組織映射圖．．．．．．．．．．．．．．．．．．．．．20
2.9 改良式自我組織映射圖．．．．．．．．．．．．．．．．．．．23
三、研究方法．．．．．．．．．．．．．．．．．．．．．．．．．28
3.1 模擬為基礎的訓練樣本產生機制．．．．．．．．．．．．．．29
3.2 資料前處理機制．．．．．．．．．．．．．．．．．．．．．29
3.3 以兩階段SOM將訓練樣本分群．．．．．．．．．．．．．．．30
3.4 以SOM為基之即時排程法則類別選擇機制．．．．．．．．．．30
3.5 以加強式學習為基之多控制器．．．．．．．．．．．．．．．32
四、案例建構與訓練樣本產生機制．．．．．．．．．．．．．．．．36
4.1 本研究半導體製造系統之問題敘述．．．．．．．．．．．．．36
4.2 本研究彈性製造系統之問題描述．．．．．．．．．．．．．．41
4.3 訓練樣本描述．．．．．．．．．．．．．．．．．．．．．．44
五、實驗與結果分析．．．．．．．．．．．．．．．．．．．．．．49
5.1 模擬實驗模式之建構與訓練樣本之產生．．．．．．．．．．．49
5.2 LVF屬性篩選結果．．．．．．．．．．．．．．．．．．．．50
5.3 兩階段SOM分群結果．．．．．．．．．．．．．．．．．．．50
5.4 本研究之智慧型多控制器績效評估．．．．．．．．．．．．．54
5.4.1 半導體製造系統實驗結果．．．．．．．．．．．．．．．．54
5.4.2 彈性製造系統實驗結果．．．．．．．．．．．．．．．．．56
六、結論與未來研究方向．．．．．．．．．．．．．．．．．．．．60
6.1 結論．．．．．．．．．．．．．．．．．．．．．．．．．．60
6.2 未來研究方向．．．．．．．．．．．．．．．．．．．．．．61
參考文獻．．．．．．．．．．．．．．．．．．．．．．．．．．．62

[1]宮大川，林恬宇，「從Melnyk 和Browne的模式建構生產活動控制系統所需的功能模組」，第八屆全國自動化科技研討會論文集(二) ，民國八十四年，第727-734頁。
[2]陳敬瑜，「使用演化式演算法最佳化彈性製造系統之生產規劃」，逢甲大學資訊工程研究所碩士論文，民國九十三年一月。
[3]王世佑，「探討流程自動化機構之模組式建構法-以彈性製造系統為例」，朝陽科技大學工業工程與管理研究所碩士論文，民國九十三年七月。
[4]王智玲，「延伸自組映射圖分群集合式類別型資料」，國立雲林科技大學資訊管理系碩士班碩士論文，2006。
[5]Jones, A. T., and Saleh, A., “A multi-level/multi-layer architecture for intelligent shop floor control.”, International Journal of Computer Integrated Manufacturing, 3, 1990, pp.60-70.
[6]Smith, J. S., Hoberecht, W. C., and Joshi, S. B., “A shop-floor control architecture for computer-integrated manufacturing.”, IIE Transactions, 28, 1996, pp.783-794.
[7]Cho, H., and Wysk, R. A., “Intelligent workstation controller for computer-integrated manufacturing: problems and models.”, Journal of Manufacturing Systems, 14, , 1995, pp.252-263.
[8]Chang, T. C., Wysk, R. A., and Wang, H. P., “Computer-Aided Manufacturing”, 2nd edn, 1998 (Prentice-Hall: Eaglewood Cliffs).
[9]Cho, H., and Wysk, R. A., “A robust adaptive scheduler for an intelligent workstation controller.”, International Journal of Production Research, 31, 1993, pp.771-789.
[10]Chiu, C., and Yih, Y., “A learning-based methodology for dynamic scheduling in distributed manufacturing systems.” International Journal of Production Research, 33, 1995, pp.3217-3232.
[11]Park, S. C., Raman, N., and Shaw, M. J., “Adaptive scheduling in dynamic flexible manufacturing systems: a dynamic rule selection approach.”, IEEE Transactions on Robotics and Automation, 13, 1997, pp.486-502.
[12]Wein, L. M., “Scheduling semiconductor wafer fabrication.”, IEEE Transactions on semiconductor manufacturing, 1, 1988, pp.115-130.
[13]Kumar, P. R., “Scheduling semiconductor manufacturing plants.”, IEEE Control Systems, December, 1994, pp.33–40.
[14]Li, S., Tang, T. and Collins, D. W., “Minimum inventory variability schedule with application in semiconductor fabrication.” IEEE Transactions on Semiconductor Manufacturing, 9, 1996, pp.145–149.
[15]Baker, K. R., “Sequencing rules and due-date assignments in a job shop.”, Management Science, 30, 1984, pp.1093-1104.
[16]Tang, L. L., Yih, Y. and Liu, C. Y., “A framework for part type selection and scheduling in FMS environments.”, International Journal of Computer Integrated Manufacturing, 8, 1995, pp.102–115
[17]Su, C. T., and Shiue, Y. R., “Inteligent scheduling controller for shop floor control systems: a hybrid genetic algorithm/decision tree learning approach.”, International Journal of Production Research, 41, 2003, pp.2619-2641.
[18]Sun, Y. L. and YIH, Y., “An intelligent controller for manufacturing cells.”, International Journal of Production Research, 34, 1996, pp.2353-2373.
[19]Priore, P., De La Fuente, D., Gomez, A. and Puente, J., “A review of machine learning in dynamic scheduling of flexible manufacturing systems.”, Artificial Intelligence for Engineering Design, Analysis & Manufacturing, 15, 2001, pp.251-263.
[20]Kohonen, T., “Self-Organizing Map third edition” (New York, NY: Springer-Verlag), 2001.
[21]Groover, M. P., “Automation, Production Systems, and Computer-Integrated Manufacturing”, Prentice-Hall, 2001.
[22]Rumelhart, D. E., Hinton, G. E., and Williams, R. J., “Learning internal representations by error propagation.”, Parallel Distributed Processing (MIT press: Cambridge) , 1986.
[23]Arzi, Y., and Iaroslavitz, L., “Neural network-based adaptive production control system for flexible manufacturing cell under a random environment”, IIE Transactions, 31, 1999, pp.217-230.
[24]Shiue, Y. R., and Guh, R. S., “Learning-based multi-pass adaptive scheduling for a dynamic manufacturing cell environment.”, Robotics and Computer-Integrated Manufacturing, 22, 2006, pp.203-216.
[25]Arzi, Y., and Iaroslavitz, L., “Operating an FMC by a decision-tree-based adaptive production control system.”, International Journal of Production Research, 38, 2000, pp.675-697.
[26]Quinlan, J. R., “Induction of decision trees,” Machine Learning, 1, 1986, pp.81-106.
[27]Quinlan, J. R., “C4.5: Programs for Machine Learning” (San Manteo: Morgan Kaufmann), 1993.
[28]Lee, C. Y., Piramuthu, S., and Tsai, Y. K., “Job shop scheduling with a genetic algorithm and machine learning.”, International Journal of Production Research, 35, 1997, pp.1171-1191.
[29]Kim, C. O., Min, H. S., and Yih, Y., “Integration of inductive learning and neural networks for multi-objective FMS scheduling.”, International Journal of Production Research, 36, 1998, pp.2497-2509.
[30]Liu, H., and Motoda, H., “Feature Selection for Knowledge Discovery and Data Mining”, (Boston: Kluwer Academic Publishers), 1998.
[31]Guajardo, J., Miranda, J., and Weber, R., “A Hybrid Forecasting Methodology using Feature Selection and Support Vector Regression.”, The Fifth International Conference on Hybrid Intelligent Systems, Rio de Janeiro, Brazil, 2005, pp.341-346.
[32]Blum, A.L., and Langley, P., “Selection of relevant features and examples in machine learning.”, Artificial Intelligence, 1997, pp.245-271.
[33]Liu, H., and Setiono R., “A Probabilistic Approach to Feature Selection – A FilterSolution”, 13th International Conf. on Machine Learning (ICML ’96), 1996 ,pp.319-327.
[34]Brassard, G., and Bratley, P., “Fundamentals of Algorithmics”, 1995, pp.353-366.
[35]Shen, W., Norrie, D. H., and Barthes, J. P. A., “Multi-Agent systems for concurrent intelligent design and manufacturing“(Taylor & Francis: London) , 2000.
[36]Wooldridge, M. J. and Jennings, N. R. “Intelligent agents: Theory and practice”, Knowledge Engineering Review, vol. 10, no. 2, 1995, pp.115-152.
[37]Sutton, R. S., and Barto, G. B., “Reinforcement Learning: An Introduction.”, (MIT Press: Cambridge), 1998.
[38]Watkins C. J. C. H., and Dayan P., “Technical note: Q learning”, Machine Learning, vol. 8, no. 3, 1992, pp. 279-292.
[39]Wang, Y. C., and John M., “Application of reinforcement learning for agent-based production scheduling.” Engineering Applications of Artificial Intelligence, 18, 2005, pp.73-82.
[40]Wang, Y. C., and John M., “Learning policies for single machine job dispatching.” Robotics and Computer-Integrated Manufacturing, 20, 2004, pp.553-562.
[41]Vesanto, J., and Alhoniemi, E., “Clustering of the Self-organizing map.”, IEEE Transactions on Neural Networks, 11(3) , 2000, pp.586-600.
[42]Davies, F. D., and Bouldin, D. W., “A cluster separation measure.”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1979, pp.224–227.
[43]Hsu, C. C., “Generalizing self-organizing map for categorical data.”, IEEE Transactions on Neural Networks, Vol. 17, No. 2, 2006, pp. 194-204
[44]Wu, S. D., and Wysk, R. A., “An application of discrete-event simulation to on-line control and scheduling in flexible manufacturing.”, International Journal of Production Research, 27, 1989, pp.1603-1623.
[45]Lu, S. C. H., Ramaswamy, D., and Kumer, P. R., “Efficient scheduling policies to reduce mean and variance of cycle-time in semiconductor manufacturing plants.”, IEEE Transactions on semiconductor manufacturing, 7, 1994, pp.374-388.
[46]Hsieh, B. W., Chen, C. H., and Chang, S. C., “Scheduling semiconductor wafer fabrication by using ordinal optimization-based simulation.”, IEEE Transactions on Robotics and Automation, 17, 2001, pp.599-608.
[47]Montazeri, M. and Wassenhove, L. N. Van, “Analysis of scheduling rules for an FMS.”, International Journal of Production Research, 28, 1990, pp.785-802.
[48]eM-Plant, “Objects Manual Version and Reference Manual 7.0 “(Tecnomatix Technologies: Stuttgart) , 2003.
[49]Chen, C. C., Yih, Y., and Wu, Y. C., “Auto-bias selection for learning-based scheduling systems”, International Journal of Production Research, 37,1999, pp.1987-2002.
[50]Su C.-T. and Shiue Y. R., “Intelligent scheduling controller for shop floor control systems: a hybrid genetic algorithm/decision tree learning approach”, International Journal of Production Research, Vol. 41, No.12, 2003, pp2619-2641.
[51]Shiue Y. R., “Data-mining-based dynamic dispatching rule selection mechanism for shop floor control systems using a support vector machine approach”, International Journal of Production Research, Vol. 41, No.12, 2008, pp1-22.