臺灣博碩士論文加值系統

開發新製程所需的時間以及成本愈來愈高，因此使用半導體工藝模擬軟體輔助貝葉 斯強化學習根據少量的資料點進行模型更新後選擇可能的最佳的製程參數進行實驗以 減少因過多嘗試次數所帶來的成本。我們以雷射退火的片電阻作為最佳化的目標，在每 一次實驗的我們會得到一組製程參數及對應結果的組合並將其加入模型的訓練集中。在 貝葉斯強化學習中，我們分別使用固定先驗函數以及可變先驗函數進行訓練。由結果可 以得知，使用固定先驗函數雖然可以得到最小值 44.32 (Ω/sq)，但是在其他超參數設定其 結果會飄移至 47.54 及 48.4 (Ω/sq)。可變先驗函數會在每一步藉由得到的資料修正先驗 函數中錯誤的部分確保錯誤的資料不會被保留至下一步。藉由此方法在不同的超參數設 定下皆可得到其結果為 44.32 及 45.19 (Ω/sq)。比起固定先驗函數，可變先驗函數更不易 受到超參數設定影響。

Develop new semiconductor processes consume more and more time and cost. Therefore, we applied Bayesian reinforcement learning with assistance of technology computer-aided design to update model by little number of datapoints and then choose possible best processing parameters to conduct experiment to reduce cost due to excessive trails in this study. We used sheet resistance of sample treated by laser annealing as target of optimization. In every trail, we received experiment parameters and its result, and then added it to training dataset. In Bayesian reinforcement learning, fixed prior and variable prior are applied. From results, although training with fixed prior can get minimum value, 44.32 (Ω/sq), there are some results shifting to 47.54 and 48.4 (Ω/sq) with different setting of hyperparameters. Variable prior can fix its data after every trail to ensure wrong data will not be keep in next trail. By this way, results are 44.32 and 45.19 (Ω/sq) with different setting of hyperparameters whose variation is less than fixed one.

摘要 i
ABSTRACT ii
Content iii
Figures list v
Tables list vi
Chapter 1 Introduction 1
1.1 Introduction of laser annealing 1
1.1.1 Small thermal budget and simple processing condition 1 1.1.2 Less dopant diffusion 1
1.1.3 Exceed equilibrium solubility and abrupt junction 2
1.2 Technology computer-aided design (TCAD) 3
1.3 Introduction of machine learning 5
1.3.1 Supervised learning (SL) 5
1.3.2 Semi-supervised learning (SSL) 7
1.3.3 Unsupervised learning (UL) 9
1.3.4 Ensemble learning 9
1.3.5 Reinforcement learning (RL) 11
1.3.6 Transfer learning 12
1.4 Machine learning methods based on Bayes’ theorem 13
1.4.1 Bayesian neural network (BNN) 14
1.4.2 Bayesian reinforcement learning (BRL)14
Chapter 2 Literature Review 16
Chapter 3 Methodology 17
3.1 Sample fabrication 17
3.2 TCAD simulation 19
3.3 Machine learning algorithms 20
3.3.1 Reinforcement learning (RL) 20
3.3.2 Bayesian reinforcement learning (BRL) 21
Chapter 4 Results and discussion 24
4.1 The result of laser annealing 24
4.1.1 Analysis of sheet resistance based on different experiment parameters 24
4.1.2 Analysis of results based on TEM and EDX 25
4.2 The result of RL and BRL 26
4.2.1 Results of RL 28
4.2.2 Results of BRL with fixed prior 30
4.2.3 Results of BRL with variable prior 32
4.2.4 Analysis of two different action 35
4.2.5 Discussion of Bayesian formula used in this study 36
4.2.6 Analysis of fixed prior and variable prior 36
4.2.7 Comparison with other methods 37
Chapter 5 Conclusion 38
Reference 39

[1] N. D. Young, J. B. Clegg, and E. A. Maydell‐Ondrusz, "Low‐temperature annealing of shallow arsenic‐implanted layers," Journal of Applied Physics, vol. 61, no. 6, pp. 2189- 2194, 1987, doi: 10.1063/1.337979.
[2] K.L. Pey and P. S. Lee, "Pulsed laser annealing technology for nanoscale fabrication of silicon-based devices in semiconductors," in Advances in Laser Materials Processing, 2 ed.: Woodhead Publishing, 2010, ch. 12, pp. 327-364.
[3] B. Rajendran et al., "Low Thermal Budget Processing for Sequential 3-D IC Fabrication," IEEE Transactions on Electron Devices, vol. 54, no. 4, pp. 707-714, 2007, doi: 10.1109/ted.2007.891300.
[4] C. White, J. Narayan, and R. Young, "Laser annealing of ion-implanted semiconductors," Science, vol. 204, no. 4392, pp. 461-468, 1979.
[5] S. Whelan et al., "Redistribution and electrical activation of ultralow energy implanted boron in silicon following laser annealing," Journal of Vacuum Science & Technology B: Microelectronics and Nanometer Structures Processing, Measurement, and Phenomena, pp. 644-649, 2002, doi: 10.1116/1.1459725.
[6] O. Gluschenkov and H. Jagannathan, "Laser Annealing in CMOS Manufacturing," ECS Transcations, pp. 11-23, 2018, doi: 10.1149/08506.0011ecst.
[7] I. Boyd and J. Wilson, "Laser annealing for semiconductor devices," Nature, vol. 287, no. 5780, pp. 278-278, 1980.
[8] Y. Takamura, S. H. Jain, P. B. Griffin, and J. D. Plummer, "Thermal stability of dopants in laser annealed silicon," Journal of Applied Physics, pp. 230-234, 2002, doi: 10.1063/1.1481975.
[9] A. L. Robinson, "Laser Annealing: Processing Semiconductors Without a Furnace," Science, vol. 201, no. 4353, pp. 333-335, 1978.
[10] L. Hess et al., "Applications of Laser Annealing in IC Fabrication," MRS Online Proceedings Library (OPL), vol. 13, 1982.
[11] A. Shima and A. Hiraiwa, "Ultra-Shallow Junction Formation by Non-Melt Laser Spike Annealing and its Application to Complementary Metal Oxide Semiconductor Devices in 65-nm Node," Japanese Journal of Applied Physics, vol. 45, no. 7R, 2006, doi: 10.1143/jjap.45.5708.
[12] P. A. Alba et al., "Nanosecond laser annealing for phosphorous activation in ultra-thin implanted silicon-on-insulator substrates," in 2016 21st International Conference on Ion Implantation Technology (IIT), 2016: IEEE, pp. 1-4.
[13] Q. Zhang et al., "Drive-Current Enhancement in Ge n-Channel MOSFET Using Laser Annealing for Source/Drain Activation," IEEE Electron Device Letters, vol. 27, no. 9, pp. 728-730, 2006, doi: 10.1109/led.2006.880655.
[14] Sentaurus Process User Guide. 690 East Middlefield Road Mountain View, CA 94043: Synopsys, Inc., 2016.
[15] J. Bell, "What is machine learning?," Machine Learning and the City: Applications in Architecture and Urban Design, pp. 207-216, 2022.
[16] D. Silver et al., "Mastering the game of Go without human knowledge," Nature, vol. 550, no. 7676, pp. 354-359, 2017/10/01 2017, doi: 10.1038/nature24270.
[17] G. Carleo et al., "Machine learning and the physical sciences," Reviews of Modern Physics, vol. 91, no. 4, p. 045002, 12/06/ 2019, doi: 10.1103/RevModPhys.91.045002.
[18] M. I. Jordan and T. M. Mitchell, "Machine learning: Trends, perspectives, and
prospects," Science, vol. 349, no. 6245, pp. 255-260, 2015, doi:
doi:10.1126/science.aaa8415.
[19] E. G. Learned-Miller, "Introduction to supervised learning," I: Department of Computer
Science, University of Massachusetts, p. 3, 2014.
[20] P. Cunningham, M. Cord, and S. J. Delany, "Supervised learning," Machine learning
techniques for multimedia: case studies on organization and retrieval, pp. 21-49, 2008.
[21] R. Saravanan and P. Sujatha, "A State of Art Techniques on Machine Learning Algorithms: A Perspective of Supervised Learning Approaches in Data Classification," in 2018 Second International Conference on Intelligent Computing and Control Systems
(ICICCS), 14-15 June 2018 2018, pp. 945-949, doi: 10.1109/ICCONS.2018.8663155.
[22] S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, "Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting," in 2018 IEEE 31st Computer Security Foundations Symposium (CSF), 9-12 July 2018 2018, pp. 268-282, doi:
10.1109/CSF.2018.00027.
[23] X. Ying, "An Overview of Overfitting and its Solutions," Journal of Physics:
Conference Series, vol. 1168, no. 2, p. 022022, 2019/02/01 2019, doi: 10.1088/1742-
6596/1168/2/022022.
[24] Y. Peng and M. H. Nagata, "An empirical overview of nonlinearity and overfitting in
machine learning using COVID-19 data," Chaos, Solitons & Fractals, vol. 139, p.
110055, 2020/10/01/ 2020, doi: https://doi.org/10.1016/j.chaos.2020.110055.
[25] C. Ha, V.-D. Tran, L. Ngo Van, and K. Than, "Eliminating overfitting of probabilistic topic models on short and noisy text: The role of dropout," International Journal of Approximate Reasoning, vol. 112, pp. 85-104, 2019/09/01/ 2019, doi:
https://doi.org/10.1016/j.ijar.2019.05.010.
[26] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout:
a simple way to prevent neural networks from overfitting," The journal of machine
learning research, vol. 15, no. 1, pp. 1929-1958, 2014.
[27] C. F. G. D. Santos and J. P. Papa, "Avoiding overfitting: A survey on regularization
methods for convolutional neural networks," ACM Computing Surveys (CSUR), vol. 54, no. 10s, pp. 1-25, 2022.
[28] H. Taud and J. F. Mas, "Multilayer Perceptron (MLP)," in Geomatic Approaches for Modeling Land Change Scenarios, M. T. Camacho Olmedo, M. Paegelow, J.-F. Mas, and F. Escobar Eds. Cham: Springer International Publishing, 2018, pp. 451-455.
[29] S. Pratik et al., "Mapping Oxidation and Wafer Cleaning to Device Characteristics Using Physics-Assisted Machine Learning," ACS Omega, vol. 7, no. 1, pp. 933-946, 2022/01/11 2022, doi: 10.1021/acsomega.1c05552.
[30] A. S. Lin, P. N. Liu, S. Pratik, Z. K. Yang, T. Rawat, and T. Y. Tseng, "RRAM Compact Modeling Using Physics and Machine Learning Hybridization," IEEE Transactions on Electron Devices, vol. 69, no. 4, pp. 1835-1841, 2022, doi: 10.1109/TED.2022.3152978.
[31] Y. W. Ho et al., "Neuroevolution-Based Efficient Field Effect Transistor Compact Device Models," IEEE Access, vol. 9, pp. 159048-159058, 2021, doi: 10.1109/ACCESS.2021.3130254.
[32] A. Lin, T. Rawat, M. H. Hsu, C. Y. Chang, H. C. Tung, and T. Y. Tseng, "Machine learning compact device models applied to optoelectronic memristor," in 2022 IEEE Photonics Conference (IPC), 13-17 Nov. 2022 2022, pp. 1-2, doi: 10.1109/IPC53466.2022.9975463.
[33] Z.-K. Yang, M.-H. Hsu, C. Y. Chang, Y.-W. Ho, P.-N. Liu, and A. Lin, "Circuit convergence study using machine learning compact models," 2021.
[34] M. F. A. Hady and F. Schwenker, "Semi-supervised Learning," in Handbook on Neural Information Processing, M. Bianchini, M. Maggini, and L. C. Jain Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 215-239.
[35] N. N. Pise and P. Kulkarni, "A Survey of Semi-Supervised Learning Methods," in 2008 International Conference on Computational Intelligence and Security, 13-17 Dec. 2008 2008, vol. 2, pp. 30-34, doi: 10.1109/CIS.2008.204.
[36] H. Yu and S. Kim, "Passive Sampling for Regression," in 2010 IEEE International Conference on Data Mining, 13-17 Dec. 2010 2010, pp. 1151-1156, doi: 10.1109/ICDM.2010.9.
[37] R. Castro, C. Kalish, R. Nowak, R. Qian, T. Rogers, and J. Zhu, "Human active learning," Advances in neural information processing systems, vol. 21, 2008.
[38] B. Settles, "Active learning," Synthesis lectures on artificial intelligence and machine learning, vol. 6, no. 1, pp. 1-114, 2012.
[39] T. S. Rawat et al., "Meta-Learned and TCAD-Assisted Sampling in Semiconductor Laser Annealing," ACS Omega, vol. 8, no. 1, pp. 737-746, 2023/01/10 2023, doi: 10.1021/acsomega.2c06000.
[40] V.-L. Nguyen, M. H. Shaker, and E. Hüllermeier, "How to measure uncertainty in uncertainty sampling for active learning," Machine Learning, vol. 111, no. 1, pp. 89- 122, 2022/01/01 2022, doi: 10.1007/s10994-021-06003-9.
[41] D. J. C. MacKay, "Information-Based Objective Functions for Active Data Selection," Neural Computation, vol. 4, no. 4, pp. 590-604, 1992, doi: 10.1162/neco.1992.4.4.590.
[42] Y. Freund, H. S. Seung, E. Shamir, and N. Tishby, "Selective Sampling Using the Query by Committee Algorithm," Machine Learning, vol. 28, no. 2, pp. 133-168, 1997/08/01 1997, doi: 10.1023/A:1007330508534.
[43] Z. Ghahramani, "Unsupervised Learning," in Advanced Lectures on Machine Learning: ML Summer Schools 2003, Canberra, Australia, February 2 - 14, 2003, Tübingen, Germany, August 4 - 16, 2003, Revised Lectures, O. Bousquet, U. von Luxburg, and G. Rätsch Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 72-112.
[44] S. Wang et al., "K-Means Clustering With Incomplete Data," IEEE Access, vol. 7, pp. 69162-69171, 2019, doi: 10.1109/ACCESS.2019.2910287.
[45] L. Badino, C. Canevari, L. Fadiga, and G. Metta, "An auto-encoder based approach to unsupervised learning of subword units," in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4-9 May 2014 2014, pp. 7634-7638, doi: 10.1109/ICASSP.2014.6855085.
[46] R. F. Mansour, J. Escorcia-Gutierrez, M. Gamarra, D. Gupta, O. Castillo, and S. Kumar, "Unsupervised Deep Learning based Variational Autoencoder Model for COVID-19 Diagnosis and Classification," Pattern Recognition Letters, vol. 151, pp. 267-274, 2021/11/01/ 2021, doi: https://doi.org/10.1016/j.patrec.2021.08.018.
[47] K. Han, Y. Wang, C. Zhang, C. Li, and C. Xu, "Autoencoder Inspired Unsupervised Feature Selection," in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 15-20 April 2018 2018, pp. 2941-2945, doi: 10.1109/ICASSP.2018.8462261.
[48] X. Dong, Z. Yu, W. Cao, Y. Shi, and Q. Ma, "A survey on ensemble learning," Frontiers of Computer Science, vol. 14, no. 2, pp. 241-258, 2020/04/01 2020, doi: 10.1007/s11704-019-8208-z.
[49] O. Sagi and L. Rokach, "Ensemble learning: A survey," WIREs Data Mining and
Knowledge Discovery, vol. 8, no. 4, p. e1249, 2018, doi: https://doi.org/10.1002/widm.1249.
[50] F. Huang, G. Xie, and R. Xiao, "Research on Ensemble Learning," in 2009 International Conference on Artificial Intelligence and Computational Intelligence, 7-8 Nov. 2009 2009, vol. 3, pp. 249-252, doi: 10.1109/AICI.2009.235.
[51] G. I. Webb and Z. Zheng, "Multistrategy ensemble learning: reducing error by combining ensemble learning techniques," IEEE Transactions on Knowledge and Data Engineering, vol. 16, no. 8, pp. 980-991, 2004, doi: 10.1109/TKDE.2004.29.
[52] M. A. Wiering and M. Van Otterlo, "Reinforcement learning," Adaptation, learning, and optimization, vol. 12, no. 3, p. 729, 2012.
[53] L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," Journal of artificial intelligence research, vol. 4, pp. 237-285, 1996.
[54] S. Kuutti, R. Bowden, H. Joshi, R. d. Temple, and S. Fallah, "End-to-end Reinforcement Learning for Autonomous Longitudinal Control Using Advantage Actor Critic with
Temporal Context," in 2019 IEEE Intelligent Transportation Systems Conference
(ITSC), 27-30 Oct. 2019 2019, pp. 2456-2462, doi: 10.1109/ITSC.2019.8917387.
[55] K. Shao, D. Zhao, N. Li, and Y. Zhu, "Learning Battles in ViZDoom via Deep Reinforcement Learning," in 2018 IEEE Conference on Computational Intelligence and
Games (CIG), 14-17 Aug. 2018 2018, pp. 1-4, doi: 10.1109/CIG.2018.8490423.
[56] K. Weiss, T. M. Khoshgoftaar, and D. Wang, "A survey of transfer learning," Journal
of Big Data, vol. 3, no. 1, p. 9, 2016/05/28 2016, doi: 10.1186/s40537-016-0043-6.
[57] J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, and G. Zhang, "Transfer learning using computational intelligence: A survey," Knowledge-Based Systems, vol. 80, pp. 14-23,
2015/05/01/ 2015, doi: https://doi.org/10.1016/j.knosys.2015.01.010.
[58] A. Gronewold and D. Vallero, "Applications of Bayes' theorem for predicting
environmental damage," ed. New York: McGraw Hill, 2010.
[59] L. V. Jospin, H. Laga, F. Boussaid, W. Buntine, and M. Bennamoun, "Hands-On
Bayesian Neural Networks—A Tutorial for Deep Learning Users," IEEE Computational Intelligence Magazine, vol. 17, no. 2, pp. 29-48, 2022, doi: 10.1109/mci.2022.3155327.
[60] C. H. Chen, P. Parashar, C. Akbar, S. M. Fu, M.-Y. Syu, and A. Lin, "Physics-Prior Bayesian Neural Networks in Semiconductor Processing," IEEE Access, vol. 7, pp. 130168-130179, 2019, doi: 10.1109/access.2019.2940130.
[61] R. Dearden, N. Friedman, and S. Russell, "Bayesian Q-learning," Aaai/iaai, vol. 1998, pp. 761-768, 1998.
[62] I. Osband, J. Aslanides, and A. Cassirer, "Randomized prior functions for deep reinforcement learning," Advances in Neural Information Processing Systems, vol. 31, 2018.
[63] M. Ghavamzadeh, S. Mannor, J. Pineau, and A. Tamar, "Bayesian Reinforcement Learning: A Survey," Foundations and Trends® in Machine Learning, vol. 8, no. 5-6, pp. 359-483, 2015, doi: 10.1561/2200000049.
[64] C. J. Hoel, K. Wolff, and L. Laine, "Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation," in 2020 IEEE Intelligent Vehicles Symposium (IV), 19 Oct.-13 Nov. 2020 2020, pp. 1563-1569, doi: 10.1109/IV47402.2020.9304614.
[65] M. Bellini, G. Pantalos, P. Kaspar, L. Knoll, and L. De-Michielis, "An Active Deep Learning Method for the Detection of Defects in Power Semiconductors," in 2021 32nd Annual SEMI Advanced Semiconductor Manufacturing Conference (ASMC), 10-12 May 2021 2021, pp. 1-5, doi: 10.1109/ASMC51741.2021.9435657.
[66] L. Cai et al., "Exploring Active Learning for Semiconductor Defect Segmentation," in 2022 IEEE International Conference on Image Processing (ICIP), 16-19 Oct. 2022 2022, pp. 1796-1800, doi: 10.1109/ICIP46576.2022.9897842.
[67] J. Shim, S. Kang, and S. Cho, "Active Learning of Convolutional Neural Network for 43

Cost-Effective Wafer Map Pattern Classification," IEEE Transactions on Semiconductor Manufacturing, vol. 33, no. 2, pp. 258-266, 2020, doi: 10.1109/TSM.2020.2974867.
[68] G. Nicolae et al., "Automatic Parameter Tuning in Finite Element Analysis of Semiconductor Packages," in 2020 International Semiconductor Conference (CAS), 7- 9 Oct. 2020 2020, pp. 41-44, doi: 10.1109/CAS50358.2020.9268036.
[69] Y. H. Lee and S. Lee, "Deep reinforcement learning based scheduling within production plan in semiconductor fabrication," Expert Systems with Applications, vol. 191, p. 116222, 2022/04/01/ 2022, doi: https://doi.org/10.1016/j.eswa.2021.116222.
[70] I. B. Park, J. Huh, J. Kim, and J. Park, "A Reinforcement Learning Approach to Robust Scheduling of Semiconductor Manufacturing Facilities," IEEE Transactions on Automation Science and Engineering, vol. 17, no. 3, pp. 1420-1431, 2020, doi: 10.1109/TASE.2019.2956762.
[71] Y. Li, J. Du, and W. Jiang, "Reinforcement Learning for Process Control with Application in Semiconductor Manufacturing," arXiv preprint arXiv:2110.11572, 2021.
[72] D. J. Pradeep and M. M. Noel, "A Finite Horizon Markov Decision Process Based Reinforcement Learning Control of a Rapid Thermal Processing system," Journal of Process Control, vol. 68, pp. 218-225, 2018/08/01/ 2018, doi:
https://doi.org/10.1016/j.jprocont.2018.06.002.
[73] T. Altenmüller, T. Stüker, B. Waschneck, A. Kuhnle, and G. Lanza, "Reinforcement
learning for an intelligent and autonomous production control of complex job-shops under time constraints," Production Engineering, vol. 14, no. 3, pp. 319-328, 2020/06/01 2020, doi: 10.1007/s11740-020-00967-8.
[74] C.-Y. Chang, C. C. Hsu, T. Rawat, S.-W. Chen, and A. Lin, Human machine competition in intelligent laser manufacturing in semiconductor processes (SPIE Optical Engineering + Applications). SPIE, 2022.
[75] A. Lin, T. Rawat, C. Y. Chang, H. C. Tung, H. L. Liu, and P. Yu, "Optical Proximity Correction Using Machine Learning Assisted Human Decision," IEEE Photonics Journal, vol. 15, no. 1, pp. 1-9, 2023, doi: 10.1109/JPHOT.2022.3231426.
[76] T. Rawat, C.-Y. Chung, S. Chen, and A. Lin, Reinforcement Learning based Intelligent Semiconductor Manufacturing Applied to Laser Annealing. 2022.
[77] J. Liu, X. Wang, S. Shen, G. Yue, S. Yu, and M. Li, "A Bayesian Q-Learning Game for Dependable Task Offloading Against DDoS Attacks in Sensor Edge Cloud," IEEE Internet of Things Journal, vol. 8, no. 9, pp. 7546-7561, 2021, doi: 10.1109/JIOT.2020.3038554.
[78] H. Kong, J. Yan, H. Wang, and L. Fan, "Energy management strategy for electric vehicles based on deep Q-learning using Bayesian optimization," Neural Computing and Applications, vol. 32, no. 18, pp. 14431-14445, 2020/09/01 2020, doi: 10.1007/s00521-019-04556-4.