臺灣博碩士論文加值系統

對於人工智慧（Artificial Intelligence）的研究，電腦遊戲是一個重要的領域。遊戲可以依據所牽涉的機率事件（Chance）兩種不同的作用分為：決定型（Deterministic）對比隨機型（Stochastic）以及完全資訊（Perfect Information）對比不完全資訊（Imperfect Information）。由於許多真實世界的問題會牽涉到不確定性，隨機型遊戲以及不完全資訊遊戲是非常值得研究的。本論文以隨機型完全資訊遊戲為研究主題，因為此類型遊戲相較於不完全資訊遊戲更容易模擬。暗棋以及一個簡化且已解的變種2×4暗棋是其中兩個此類型的遊戲，也是本論文主要研究的遊戲。
本論文首先使用數個現存之結合額外知識的技術強化一支基於蒙地卡羅樹搜尋（Monte-Carlo Tree Search）的暗棋程式。結合於此暗棋程式中的額外知識是人為設定的，並且被併入以下四個技術：提早模擬結束（Early Playout Terminations）、隱性最小極大更新（Implicit Minimax Backups）、基於品質的獎勵值（Quality-Based Rewards）、以及先進偏差（Progressive Bias）。進一步結合這四個技術的程式對原程式達到84.75%（±1.90%）的勝率。
此外，本論文透過2×4暗棋研究三種強度分析度量（Strength Analysis Metric），分別為對其他玩家的勝率、對專家走步（Expert Action）的預測率（Prediction Rate）、以及對盤面價值（Value of Position）的均方差（Mean Squared Error）。實驗結果顯示，勝率的確是衡量程式強度一種好的指標。另外兩種度量雖然不如勝率，但也是衡量程式強度不錯的指標。另一個在2×4暗棋上進行研究的是阿法零演算法（AlphaZero Algorithm）。阿法零演算法是一種強化學習（Reinforcement Learning）演算法，已經在西洋棋、日本將棋、以及圍棋上達到超越人類的水準。實驗結果顯示，即使是隨機型遊戲，此演算法也能夠習得盤面的理論值（Theoretical Value）以及最佳下法（Optimal Play）。
最後，此論文也研究另外兩個隨機型完全資訊遊戲：愛因斯坦棋（EinStein Würfelt Nicht!）以及2048類型遊戲。另一種強化學習演算法時序差異學習（Temporal Difference Learning）被應用到這兩個遊戲。在愛因斯坦棋中，結合下列三種使用習得知識之技術的程式對原程式有62.25%（±2.12%）的勝率。此三種技術分別為先進偏差、先備知識（Prior Knowledge）、以及 ε-貪心模擬（Epsilon-Greedy Playouts）。對於2048類型遊戲，一個多階段（Multistage）的時序差異學習變種則用來增強學習到的知識。

The field of computer games is important to the researches in artificial intelligence. According to two different roles of the elements of chance involved, games can be classified as deterministic vs. stochastic and perfect information vs. imperfect information. Since many real-world problems involve uncertainty, stochastic games and imperfect information games are worthy to study. This thesis targets at stochastic games with perfect information since the games in this category is easier to model than imperfect information games. Chinese dark chess (CDC) and a reduced and solved variant, 2×4 CDC, are two games of this category which this thesis mainly focuses on.
This thesis first enhances a game-playing program for CDC based on Monte-Carlo tree search (MCTS) by several existing techniques that combine additional knowledge. The additional knowledge is manually designed, and is incorporated into four techniques including early playout terminations, implicit minimax backups, quality-based rewards, and progressive bias. By combining all, the win rate is 84.75% (±1.90%) against the original program.
In addition, this thesis investigates three strength analysis metrics on 2×4 CDC, including win rates playing against other players, prediction rates to expert actions, and mean squared errors to values of positions. Experiments show that win rates are indeed good indicators of programs’ strengths. The other two metrics are also good indicators, though not as good as win rates. Another analysis performed on 2×4 CDC is applying the AlphaZero algorithm, which is a kind of reinforcement learning algorithm achieved superhuman levels of plays in chess, shogi, and Go. Experiments show that the algorithm can learn the theoretical values and optimal plays even in stochastic games.
Finally, this thesis studies two more stochastic games with perfect information, which are EinStein Würfelt Nicht! (EWN) and 2048-like games. Another kind of reinforcement learning algorithm, temporal difference learning, is applied to EWN and 2048-like games. For EWN, a program combining three techniques using the learned knowledge, including progressive bias, prior knowledge, and epsilon-greedy playouts, has a win rate of 62.25% (±2.12%) against the original program. For 2048-like games, a multistage variant of temporal difference learning improves the learned knowledge.

摘要 i
Abstract iii
致謝 v
List of Figures x
List of Tables xii
Chapter 1 Introduction 1
1.1 Chinese Dark Chess and a Reduced Variant 4
1.1.1 Chinese dark chess (CDC) 4
1.1.2 2×4 Chinese Dark Chess (2×4 CDC) 6
1.2 Motivations and Goals 7
1.3 Organization 9
Chapter 2 Strength Improvements for CDC Programs 10
2.1 Monte-Carlo Tree Search 11
2.2 Computer CDC 12
2.3 Incorporated Techniques 13
2.3.1 Early Playout Terminations 13
2.3.2 Implicit Minimax Backups 14
2.3.3 Quality-Based Rewards 14
2.3.4 Progressive Bias 16
2.4 Incorporation into CDC 17
2.4.1 Early Playout Terminations 17
2.4.2 Implicit Minimax Backups 18
2.4.3 Quality-Based Rewards 19
2.4.4 Progressive Bias 20
2.5 Experiments 22
2.5.1 Individual Techniques 24
2.5.1.1 Early Playout Terminations 24
2.5.1.2 Implicit Minimax Backups 25
2.5.1.3 Quality-Based Rewards 28
2.5.1.4 Progressive Bias 30
2.5.2 Combinations of Techniques 35
2.6 Fine Tunings for Tournaments and Results 37
2.7 Chapter Conclusions 38
Chapter 3 Strength Analysis Metrics for CDC Programs 40
3.1 Investigated Strength Analysis Metrics 41
3.1.1 Win Rates Playing against a Designated Player 41
3.1.2 Prediction Rates to Expert Actions 43
3.1.3 Mean Squared Errors to Values of Positions 44
3.2 Simple Linear Regression 45
3.3 Experiments on 2×4 CDC 46
3.3.1 Win Rates 48
3.3.2 Prediction Rates 51
3.3.3 Mean Squared Errors 54
3.4 Experiments on Predicting Win Rates from Smaller Variants 58
3.5 Chapter Conclusions 60
Chapter 4 Applying AlphaZero to 2×4 CDC 62
4.1 AlphaZero Algorithm 63
4.2 Tabular AlphaZero for 2×4 CDC 65
4.3 Experiments 70
4.3.1 Experiment Settings 70
4.3.2 Symmetries of Positions 72
4.3.3 cpuct 74
4.3.4 Dirichlet α 77
4.3.5 Dirichlet ε 78
4.3.6 Temperature τ 80
4.3.7 Discussions on Four Tested Hyper-parameters 82
4.4 Chapter Conclusions 83
Chapter 5 More Stochastic Games 85
5.1 Temporal Difference Learning 85
5.2 Investigation on EinStein Würfelt Nicht! 87
5.2.1 EinStein Würfelt Nicht! 87
5.2.2 Incorporating N-tuple Networks into MCTS 88
5.2.3 Results 89
5.3 Investigation on 2048-Like Games 89
5.3.1 2048-Like Games 90
5.3.1.1 2048 90
5.3.1.2 Threes 92
5.3.2 Multistage Temporal Difference Learning 93
5.3.3 Results 94
5.4 Chapter Conclusions 95
Chapter 6 Conclusions and Future Research Directions 96
6.1 Conclusions 96
6.2 Future Research Directions 98
References 101
Appendix A. Rules for Early Playout Terminations 116
Appendix B. Detailed Results of the Strength Analysis Metrics 118
Appendix C. Detailed Results of AlphaZero on 2×4 CDC 128
Curriculum Vitae 156

1. Althöfer, I., On the Origins of "EinStein würfelt nicht!", retrieved Mar. 5, 2019, from: https://althofer.de/origins-of-ewn.html.
2. Althöfer, I., “Hanfried wins EinStein Würfelt Nicht! tournament”, ICGA Journal, 38(4), pp. 246, 2015, doi: 10.3233/ICG-2015-38408.
3. Asuero, A. G., Sayago, A., and González, A. G., “The correlation coefficient: An overview”, Critical Reviews in Analytical Chemistry, 36(1), pp. 41-59, 2006, doi: 10.1080/10408340500526766.
4. Auer, P., Cesa-Bianchi, N., and Fischer, P., “Finite-time analysis of the multiarmed bandit problem”, Machine Learning, 47(2-3), pp. 235-256, 2002, doi: 10.1023/A:1013689704352.
5. Baba, T. and Ito, T. “Improvement and evaluation of search algorithm in computer dice-shogi”, in Proceedings of 2017 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2017), Taipei, Taiwan, Dec. 1-3, 2017, IEEE, pp. 84-87, doi: 10.1109/TAAI.2017.49.
6. Bagheri, S., Thill, M., Koch, P., and Konen, W., “Online adaptable learning rates for the game Connect-4”, IEEE Transactions on Computational Intelligence and AI in Games, 8(1), pp. 33-42, 2016, doi: 10.1109/TCIAIG.2014.2367105.
7. Baier, H., Sattaur, A., Powley, E. J., Devlin, S., Rollason, J., and Cowling, P. I., “Emulating human play in a leading mobile card game”, IEEE Transactions on Games, in press, doi: 10.1109/TG.2018.2835764.
8. Baier, H. and Winands, M. H. M. “Monte-Carlo tree search and minimax hybrids with heuristic evaluation functions”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2013, Cazenave, T., Winands, M. H. M., and Björnsson, Y. (Eds.), Springer International Publishing, pp. 45-63, doi: 10.1007/978-3-319-14923-3_4.
9. Baier, H. and Winands, M. H. M., “Time management for Monte Carlo tree search”, IEEE Transactions on Computational Intelligence and AI in Games, 8(3), pp. 301-314, 2016, doi: 10.1109/TCIAIG.2015.2443123.
10. Barto, A. G. and Mahadevan, S., “Recent advances in hierarchical reinforcement learning”, Discrete Event Dynamic Systems, 13(1-2), pp. 41-77, 2003, doi: 10.1023/A:1022140919877.
11. Baum, S. D., “A survey of artificial general intelligence projects for ethics, risk, and policy”, Global Catastrophic Risk Institute Working Paper 17-1, pp. 1-99, 2017, doi: 10.2139/ssrn.3070741.
12. Baxter, J., Tridgell, A., and Weaver, L., “Learning to play chess using temporal differences”, Machine Learning, 40(3), pp. 243-263, 2000, doi: 10.1023/A:1007634325138.
13. Beal, D. F. and Smith, M. C. “First results from using temporal difference learning in shogi”, in Proceedings of The 1st Conference on Computers and Games (CG'98), Tsukuba, Japan, Nov. 11-12, 1998, van den Herik, H. J. and Iida, H. (Eds.), Springer Berlin Heidelberg, pp. 113-125, doi: 10.1007/3-540-48957-6_7.
14. Beal, D. F. and Smith, M. C., “Temporal difference learning applied to game playing and the results of application to shogi”, Theoretical Computer Science, 252(1), pp. 105-119, 2001, doi: 10.1016/S0304-3975(00)00078-5.
15. Bishop, C. M., Pattern Recognition and Machine Learning, 1st edition, Springer-Verlag New York, 738 pages, 2006.
16. Bjornsson, Y. and Finnsson, H., “CadiaPlayer: A simulation-based general game player”, IEEE Transactions on Computational Intelligence and AI in Games, 1(1), pp. 4-15, 2009, doi: 10.1109/TCIAIG.2009.2018702.
17. Borsboom, J., Saito, J.-T., Chaslot, G. M. J. B., and Uiterwijk, J. W. H. M. “A comparison of Monte-Carlo methods for phantom Go”, in The 19th Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2007), Utrecht, the Netherlands, Nov. 5-6, 2008 of Conference.
18. Browne, C. B., Powley, E. J., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., . . . Colton, S., “A survey of Monte Carlo tree search methods”, IEEE Transactions on Computational Intelligence and AI in Games, 4(1), pp. 1-43, 2012, doi: 10.1109/TCIAIG.2012.2186810.
19. Buro, M., “The Othello match of the year: Takeshi Murakami vs. Logistello”, ICCA Journal, 20(3), pp. 189-193, 1997, doi: ICG-1997-20311.
20. Campbell, M., Hoane, A. J., and Hsu, F.-h., “Deep Blue”, Artificial Intelligence, 134(1), pp. 57-83, 2002, doi: 10.1016/S0004-3702(01)00129-1.
21. Cazenave, T. “A phantom-Go program”, in Proceedings of The 11th International Conference on Advances in Computer Games (ACG 2005), Taipei, Taiwan, Sep. 6-9, 2005, van den Herik, H. J., Hsu, S.-C., Hsu, T.-s., and Donkers, H. H. L. M. (Eds.), Springer Berlin Heidelberg, pp. 120-125, doi: 10.1007/11922155_9.
22. Chang, H.-J., Chen, J.-C., Hsueh, C.-W., and Hsu, T.-s., “Analysis and efficient solutions for 2×4 Chinese dark chess”, ICGA Journal, 40(2), pp. 61-76, 2018, doi: 10.3233/ICG-180049.
23. Chang, H.-J., Fan, G.-Y., Chen, J.-C., Hsueh, C.-W., and Hsu, T.-s. “Validating and fine-tuning of game evaluating functions using endgame databases”, in Proceedings of The 6th Workshop on Computer Games (CGW 2017), Held in Conjunction with the 26th International Conference on Artificial Intelligence (IJCAI-17), Melbourne, Australia, Aug. 20, 2017, Cazenave, T., Winands, M. H. M., and Saffidine, A. (Eds.), Springer International Publishing, pp. 137-150, doi: 10.1007/978-3-319-75931-9_10.
24. Chang, H.-J. and Hsu, T.-s. “A quantitative study of 2×4 Chinese dark chess”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2013, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 151-162, doi: 10.1007/978-3-319-09165-5_13.
25. Chang, H.-J., Hsueh, C.-W., and Hsu, T.-s. “Convergence and correctness analysis of Monte-Carlo tree search algorithms: A case study of 2 by 4 Chinese dark chess”, in Proceedings of 2015 IEEE Conference on Computational Intelligence and Games (CIG 2015), Tainan, Taiwan, Aug. 31-Sep. 2, 2015, IEEE, pp. 260-266, doi: 10.1109/CIG.2015.7317963.
26. Chaslot, G. M. J. B., “Monte Carlo tree search”, Department of Knowledge Engineering, Universiteit Maastricht, Ph.D. dissertation, 144 pages, 2010, Maastricht, the Netherlands.
27. Chaslot, G. M. J. B., Fiter, C., Hoock, J.-B., Rimmel, A., and Teytaud, O. “Adding expert knowledge and exploration in Monte-Carlo tree search”, in Proceedings of The 12th International Conference on Advances in Computer Games (ACG 2009), Pamplona, Spain, May 11-13, 2009, van den Herik, H. J. and Spronck, P. (Eds.), Springer Berlin Heidelberg, pp. 1-13, doi: 10.1007/978-3-642-12993-3_1.
28. Chaslot, G. M. J. B., Winands, M. H. M., van den Herik, H. J., Uiterwijk, J. W. H. M., and Bouzy, B., “Progressive strategies for Monte Carlo tree search”, New Mathematics and Natural Computation, 4(3), pp. 343-357, 2008, doi: 10.1142/S1793005708001094.
29. Chen, B.-N. and Hsu, T.-s. “Automatic generation of opening books for dark chess”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2013, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 221-232, doi: 10.1007/978-3-319-09165-5_19.
30. Chen, B.-N., Shen, B.-J., and Hsu, T.-s., “Chinese dark chess”, ICGA Journal, 33(2), pp. 93-106, 2010, doi: 10.3233/ICG-2010-33204.
31. Chen, J.-C., Fan, G.-Y., Chang, H.-J., and Hsu, T.-s., “Compressing Chinese dark chess endgame databases by deep learning”, IEEE Transactions on Games, 10(4), pp. 413-422, 2018, doi: 10.1109/TG.2018.2802484.
32. Chen, J.-C., Lin, T.-Y., Chen, B.-N., and Hsu, T.-s., “Equivalence classes in Chinese dark chess endgames”, IEEE Transactions on Computational Intelligence and AI in Games, 7(2), pp. 109-122, 2015, doi: 10.1109/TCIAIG.2014.2317832.
33. Chen, J.-C., Lin, T.-Y., Hsu, S.-C., and Hsu, T.-s. “Design and implementation of computer Chinese dark chess endgame database”, in TCGA Workshop 2012, Hualien, Taiwan, Jun. 30-Jul. 1, 2012 of Conference. (Chinese)
34. Chu, Y.-J. R., Chen, Y.-H., Hsueh, C.-H., and Wu, I.-C. “An agent for EinStein Würfelt Nicht! using n-tuple networks”, in Proceedings of 2017 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2017), Taipei, Taiwan, Dec. 1-3, 2017, IEEE, pp. 184-189, doi: 10.1109/TAAI.2017.32.
35. Ciancarini, P. and Favini, G. P., “Monte Carlo tree search in Kriegspiel”, Artificial Intelligence, 174(11), pp. 670-684, 2010, doi: 10.1016/j.artint.2010.04.017.
36. Cirulli, G., 2048, retrieved Mar. 5, 2019, from: https://play2048.co/.
37. Coulom, R. “Computing elo ratings of move patterns in the game of Go”, in Proceedings of Computer Games Workshop 2007 (CGW 2007), Amsterdam, the Netherlands, Jun. 15-17, 2007, van den Herik, H. J., Uiterwijk, J. W. H. M., Winands, M. H. M., and Schadd, M. P. D. (Eds.), pp. 113-124.
38. Cowling, P. I., Powley, E. J., and Whitehouse, D., “Information set Monte Carlo tree search”, IEEE Transactions on Computational Intelligence and AI in Games, 4(2), pp. 120-143, 2012, doi: 10.1109/TCIAIG.2012.2200894.
39. Demediuk, S., Tamassia, M., Raffe, W. L., Zambetta, F., Li, X., and Mueller, F. “Monte Carlo tree search based algorithms for dynamic difficulty adjustment”, in Proceedings of 2017 IEEE Conference on Computational Intelligence and Games (CIG 2017), New York, USA, Aug. 22-25, 2017, IEEE, pp. 53-59, doi: 10.1109/CIG.2017.8080415.
40. Došilović, F. K., Brčić, M., and Hlupić, N. “Explainable artificial intelligence: A survey”, in Proceedings of The 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO 2018), Opatija, Croatia, May 21-25, 2018, Skala, K., Koricic, M., Galinac Grbac, T., Cicin-Sain, M., Sruk, V., Ribaric, S., . . . Janjic, M. (Eds.), IEEE, pp. 210-215, doi: 10.23919/MIPRO.2018.8400040.
41. Enzenberger, M. and Müller, M. “A lock-free multithreaded Monte-Carlo tree search algorithm”, in Proceedings of The 12th International Conference on Advances in Computer Games (ACG 2009), Pamplona, Spain, May 11-13, 2009, van den Herik, H. J. and Spronck, P. (Eds.), Springer Berlin Heidelberg, pp. 14-20, doi: 10.1007/978-3-642-12993-3_2.
42. Enzenberger, M., Müller, M., Arneson, B., and Segal, R., “Fuego—An open-source framework for board games and Go engine based on Monte Carlo tree search”, IEEE Transactions on Computational Intelligence and AI in Games, 2(4), pp. 259-270, 2010, doi: 10.1109/TCIAIG.2010.2083662.
43. Facebook, ELF | Game Research Platform | Facebook AI, retrieved Mar. 5, 2019, from: https://facebook.ai/developers/tools/elf.
44. Ferguson, T. S., “A Bayesian analysis of some nonparametric problems”, The Annals of Statistics, 1(2), pp. 209-230, 1973, doi: 10.1214/aos/1176342360.
45. Finnsson, H. “Generalized Monte Carlo tree search extensions for general game playing”, in Proceedings of The 26th AAAI Conference on Artificial Intelligence (AAAI-12), Toronto, Canada, Jul. 22-26, 2012, AAAI Press, pp. 1550-1556.
46. Gao, C., Hayward, R. B., and Müller, M., “Move prediction using deep convolutional neural networks in Hex”, IEEE Transactions on Games, 10(4), pp. 336-343, 2018, doi: 10.1109/TG.2017.2785042.
47. Gao, C., Müller, M., and Hayward, R. B. “Focused depth-first proof number search using convolutional neural networks for the game of Hex”, in Proceedings of The 26th International Joint Conference on Artificial Intelligence (IJCAI-17), Melbourne, Australia, Aug. 19-25, 2017, Sierra, C. (Ed.), IJCAI Organization, pp. 3668-3674, doi: 10.24963/ijcai.2017/513.
48. Gelly, S. and Silver, D. “Combining online and offline knowledge in UCT”, in Proceedings of The 24th International Conference on Machine Learning (ICML '07), Corvallis, Oregon, USA, Jun. 20-24, 2007, Ghahramani, Z. (Ed.), ACM, pp. 273-280, doi: 10.1145/1273496.1273531.
49. Gelly, S. and Silver, D., “Monte-Carlo tree search and rapid action value estimation in computer Go”, Artificial Intelligence, 175(11), pp. 1856-1875, 2011, doi: 10.1016/j.artint.2011.03.007.
50. Gelly, S., Wang, Y., Munos, R., and Teytaud, O., “Modification of UCT with patterns in Monte-Carlo Go”, Technical Report: RR-6062, INRIA, France, 2006.
51. Han, J. and Moraga, C. “The influence of the sigmoid function parameters on the speed of backpropagation learning”, in Proceedings of The International Workshop on Artificial Neural Networks (IWANN '95): From Natural to Artificial Neural Computation, Torremolinos, Spain, Jun. 7-9, 1995, Mira, J. and Sandoval, F. (Eds.), Springer Berlin Heidelberg, pp. 195-201, doi: 10.1007/3-540-59497-3_175.
52. Henderson, P., Arneson, B., and Hayward, R. B. “Solving 8×8 Hex”, in Proceedings of The 21st International Joint Conference on Artificial Intelligence (IJCAI-09), Pasadena, California, USA, Jul. 11-17, 2009, IJCAI Organization, pp. 505-510.
53. Hsueh, C.-H. and Wu, I.-C., “DarkKnight wins Chinese dark chess tournament”, ICGA Journal, 38(4), pp. 249-251, 2015, doi: 10.3233/ICG-2015-38411.
54. Hsueh, C.-H., Wu, I.-C., Chen, J.-C., and Hsu, T.-s. “AlphaZero for a non-deterministic game”, in Proceedings of 2018 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2018), Taichung, Taiwan, Nov. 30-Dec. 2, 2018, IEEE, pp. 116-121.
55. Hsueh, C.-H., Wu, I.-C., Hsu, T.-s., and Chen, J.-C., “An investigation of strength analysis metrics for game-playing programs: A case study in Chinese dark chess”, ICGA Journal, 40(2), pp. 77-104, 2018, doi: 10.3233/ICG-180046.
56. Hsueh, C.-H., Wu, I.-C., Tseng, W.-J., Yen, S.-J., and Chen, J.-C. “Strength improvement and snalysis for an MCTS-based Chinese dark chess program”, in Proceedings of The 14th International Conference on Advances in Computer Games (ACG 2015), Leiden, the Netherlands, Jul. 1-3, 2015, Plaat, A., van den Herik, H. J., and Kosters, W. (Eds.), Springer International Publishing, pp. 29-40, doi: 10.1007/978-3-319-27992-3_4.
57. Hsueh, C.-H., Wu, I.-C., Tseng, W.-J., Yen, S.-J., and Chen, J.-C., “An analysis for strength improvement of an MCTS-based program playing Chinese dark chess”, Theoretical Computer Science, 644, pp. 63-75, 2016, doi: 10.1016/j.tcs.2016.06.025.
58. Huang, S.-C., “New heuristics for Monte Carlo tree search applied to the game of Go”, Department of Computer Science and Information Engineering, National Taiwan Normal University, Ph.D. dissertation, 91 pages, 2011, Taipei, Taiwan.
59. Hunicke, R. and Chapman, V. “AI for dynamic difficulty adjustment in games”, in Proceedings of AAAI-04 workshop on Challenges in Game Artificial Intelligence, San Jose, California, USA, Jul. 25-26, 2004, AAAI Press, pp. 91-96.
60. ICGA, Results (Computer Olympiad 2016) | ICGA, retrieved Mar. 5, 2019, from: https://icga.org/?page_id=1764.
61. ICGA, Results (Computer Olympiad 2017) | ICGA, retrieved Mar. 5, 2019, from: https://icga.org/?page_id=2050.
62. Ikeda, K., Shishido, T., and Viennot, S. “Machine-learning of shape names for the game of Go”, in Proceedings of The 14th International Conference on Advances in Computer Games (ACG 2015), Leiden, the Netherlands, Jul. 1-3, 2015, Plaat, A., van den Herik, H. J., and Kosters, W. (Eds.), Springer International Publishing, pp. 247-259, doi: 10.1007/978-3-319-27992-3_22.
63. Ikeda, K. and Viennot, S. “Efficiency of static knowledge bias in Monte-Carlo tree search”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2013, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 26-38, doi: 10.1007/978-3-319-09165-5_3.
64. Ikeda, K. and Viennot, S. “Production of various strategies and position control for Monte-Carlo Go — Entertaining human players”, in Proceedings of 2013 IEEE Conference on Computational Inteligence in Games (CIG 2013), Niagara Falls, ON, Canada, Aug. 11-13, 2013, IEEE, pp. 145-152, doi: 10.1109/CIG.2013.6633625.
65. Ishihara, M., Ito, S., Ishii, R., Harada, T., and Thawonmas, R. “Monte-Carlo tree search for implementation of dynamic difficulty adjustment fighting game AIs having believable behaviors”, in Proceedings of 2018 IEEE Conference on Computational Intelligence and Games (CIG 2018), Maastricht, the Netherlands, Aug. 14-17, 2018, IEEE, pp. 46-53, doi: 10.1109/CIG.2018.8490376.
66. Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W. M., Donahue, J., Razavi, A., . . . Kavukcuoglu, K., “Population based training of neural networks”, CoRR, abs/1711.09846, 2017.
67. Jaśkowski, W., “Systematic n-tuple networks for Othello position evaluation”, ICGA Journal, 37(2), pp. 85-96, 2014, doi: 10.3233/ICG-2014-37203.
68. Jaśkowski, W., “Mastering 2048 with delayed temporal coherence learning, multistage weight promotion, redundant encoding, and carousel shaping”, IEEE Transactions on Games, 10(1), pp. 3-14, 2018, doi: 10.1109/TCIAIG.2017.2651887.
69. Jouandeau, N. “Varying complexity in CHINESE DARK CHESS stochastic game”, in TCGA Workshop 2014, Taipei, Taiwan, Jun. 27-28, 2014 of Conference.
70. Jouandeau, N. and Cazenave, T. “Monte-Carlo tree reductions for stochastic games”, in Proceedings of 2014 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2014), Taipei, Taiwan, Nov. 21-23, 2014, Cheng, S.-M. and Day, M.-Y. (Eds.), Springer International Publishing, pp. 228-238, doi: 10.1007/978-3-319-13987-6_22.
71. Jouandeau, N. and Cazenave, T. “Small and Large MCTS Playouts Applied to Chinese Dark Chess Stochastic Game”, in Proceedings of The 3rd Workshop on Computer Games (CGW 2014), Held in Conjunction with the 21st European Conference on Artificial Intelligence (ECAI 2014), Prague, Czech Republic, Aug. 18, 2014, Cazenave, T., Winands, M. H. M., and Björnsson, Y. (Eds.), Springer International Publishing, pp. 78-89, doi: 10.1007/978-3-319-14923-3_6.
72. Kishimoto, A. and Müller, M., “Game solvers”, in Handbook of Digital Games and Entertainment Technologies (Ch. 1), Nakatsu, R., Rauterberg, M., and Ciancarini, P. (Eds), Springer Singapore, Singapore, pp. 3-22, 2015, doi: 10.1007/978-981-4560-52-8_35-1.
73. Kocsis, L. and Szepesvári, C. “Bandit based Monte-Carlo planning”, in Proceedings of The 17th European Conference on Machine Learning (ECML 2006), Berlin, Germany, Sep. 18-22, 2006, Fürnkranz, J., Scheffer, T., and Spiliopoulou, M. (Eds.), Springer-Verlag Berlin Heidelberg, pp. 282-293, doi: 10.1007/11871842_29.
74. Krawiec, K. and Szubert, M. G. “Learning n-tuple networks for Othello by coevolutionary gradient search”, in Proceedings of The 13th Annual Conference on Genetic and Evolutionary Computation (GECCO '11), Dublin, Ireland, Jul. 12-16, 2011, Krasnogor, N. (Ed.), ACM, pp. 355-362, doi: 10.1145/2001576.2001626.
75. Lai, M., “Giraffe: Using deep reinforcement learning to play chess”, Department of Computing, Imperial College London, Master dissertation, 39 pages, 2015, London, UK.
76. Lanctot, M., Winands, M. H. M., Pepels, T., and Sturtevant, N. R. “Monte Carlo tree search with heuristic evaluations using implicit minimax backups”, in Proceedings of 2014 IEEE Conference on Computational Intelligence and Games (CIG 2014), Dortmund, Germany, Aug. 26-29, 2014, IEEE, doi: 10.1109/CIG.2014.6932903.
77. Lane, D. M., Scott, D., Hebl, M., Guerra, R., Osherson, D., and Zimmer, H., Introduction to Statistics, ed. Lane, D. M., Rice University, Houston, Texas, USA, 692 pages, 2014.
78. Leela Zero, Leela Zero - GitHub, retrieved Mar. 5, 2019, from: https://github.com/leela-zero.
79. Liang, H.-M., “Study on the property of double king dark chess”, Department of Computer Science and Information Engineering, National Dong Hwa University, Master dissertation, 56 pages, 2014, Hualien, Taiwan. (Chinese)
80. Lin, S.-S., Chen, C.-H., Hsu, S.-C., Wu, I.-C., Yen, S.-J., and Chen, J.-C., “TCGA 2014 Computer Game Tournament”, ICGA Journal, 37(4), pp. 226-229, 2014, doi: 10.3233/ICG-2014-37408.
81. Lin, Y.-S., Wu, I.-C., and Yen, S.-J., “TAAI 2011 computer-game tournaments”, ICGA Journal, 34(4), pp. 248-250, 2011, doi: 10.3233/ICG-2011-34413.
82. Liskowski, P., Jaśkowski, W., and Krawiec, K., “Learning to play Othello with deep neural networks”, IEEE Transactions on Games, 10(4), pp. 354-364, 2018, doi: 10.1109/TG.2018.2799997.
83. Lorentz, R. J. “Amazons discover Monte-Carlo”, in Proceedings of The 6th International Conference on Computers and Games (CG 2008), Beijing, China, Sep. 29-Oct. 1, 2008, van den Herik, H. J., Xu, X., Ma, Z., and Winands, M. H. M. (Eds.), Springer Berlin Heidelberg, pp. 13-24, doi: 10.1007/978-3-540-87608-3_2.
84. Lorentz, R. J. “An MCTS program to play EinStein Würfelt Nicht!”, in Proceedings of The 13th International Conference on Advances in Computer Games (ACG 2011), Tilburg, the Netherlands, Nov. 20-22, 2011, van den Herik, H. J. and Plaat, A. (Eds.), Springer Berlin Heidelberg, pp. 52-59, doi: 10.1007/978-3-642-31866-5_5.
85. Lorentz, R. J., “Using evaluation functions in Monte-Carlo tree search”, Theoretical Computer Science, 644, pp. 106-113, 2016, doi: 10.1016/j.tcs.2016.06.026.
86. Lorentz, R. J. and Horey, T. “Programming Breakthrough”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2014, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 49-59, doi: 10.1007/978-3-319-09165-5_5.
87. Lucas, S. M., “Learning to play Othello with n-tuple systems”, Australian Journal of Intelligent Information Processing, 4, pp. 1-20, 2008.
88. Matsuzaki, K. “Systematic selection of n-tuple networks with consideration of interinfluence for game 2048”, in Proceedings of 2016 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2016), Hsinchu, Taiwan Nov. 25-27, 2016, IEEE, pp. 186-193, doi: 10.1109/TAAI.2016.7880154.
89. Mei, S., Montanari, A., and Nguyen, P.-M., “A mean field view of the landscape of two-layer neural networks”, Proceedings of the National Academy of Sciences of the United States of America, 115(33), pp. E7665-E7671, 2018, doi: 10.1073/pnas.1806579115.
90. Nakamichi, T. and Ito, T., “Adjusting the evaluation function for weakening the competency level of a computer shogi program”, ICGA Journal, 40(1), pp. 15-31, 2018, doi: 10.3233/ICG-180042.
91. NDHU Artificial Intelligence Laboratory, TAAI Computer Game Tournaments, retrieved Mar. 5, 2019, from: https://www.tcga.tw/taai2015/zh_TW/#tabout.
92. NDHU Artificial Intelligence Laboratory, TAAI Computer Game Tournaments 2017, retrieved Mar. 5, 2019, from: https://www.tcga.tw/taai2017/zh_TW/#tabout.
93. NDHU Artificial Intelligence Laboratory, TCGA Computer Game Tournaments 2017, retrieved Mar. 5, 2019, from: https://www.tcga.tw/tournament2017/en/#ranklist.
94. NDHU Artificial Intelligence Laboratory, ICGA Computer Olympiad 2018, retrieved Mar. 5, 2019, from: https://www.tcga.tw/icga-computer-olympiad-2018/en/.
95. NDHU Artificial Intelligence Laboratory, TAAI Computer Game Tournaments 2018, retrieved Mar. 5, 2019, from: https://www.tcga.tw/taai2018/en/#ranklist.
96. Nguyen, H., Viennot, S., and Ikeda, K. “Fast optimization of the pattern shapes in board games with simulated annealing”, in Proceedings of The 6th International Conference on Knowledge and Systems Engineering (KSE 2014), Hanoi, Vietnam, Oct. 9-11, 2014, Nguyen, V.-H., Le, A.-C., and Huynh, V.-N. (Eds.), Springer International Publishing, pp. 325-337, doi: 10.1007/978-3-319-11680-8_26.
97. Nijssen, J. A. M. and Winands, M. H. M. “Enhancements for multi-player Monte-Carlo tree search”, in Proceedings of The 7th International Conference on Computers and Games (CG 2010), Kanazawa, Japan, Sep. 24-26, 2010, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer Berlin Heidelberg, pp. 238-249, doi: 10.1007/978-3-642-17928-0_22.
98. Oka, K. and Matsuzaki, K. “Systematic selection of n-tuple networks for 2048”, in Proceedings of The 9th International Conference on Computers and Games (CG 2016), Leiden, the Netherlands, Jun. 29-Jul. 1, 2016, Plaat, A., Kosters, W., and van den Herik, H. J. (Eds.), Springer International Publishing, pp. 81-92, doi: 10.1007/978-3-319-50935-8_8.
99. Osborne, M. J. and Rubinstein, A., A Course in Game Theory, 1st edition, MIT Press, Cambridge, Massachusetts, USA, 368 pages, 1994.
100. Oster, E., “An MCTS agent for EinStein Würfelt Nicht!”, Department of Knowledge Engineering, Maastricht University, Master dissertation, 41 pages, 2015, Maastricht, the Netherlands.
101. Paulsen, P. and Fürnkranz, J. “A moderately successful attempt to train chess evaluation functions of different strengths”, in The ICML-10 Workshop on Machine Learning and Games, Haifa, Israel, Jun. 25, 2010 of Conference, Thurau, C., Driessens, K., and Missura, O. (Eds.).
102. Pawlewicz, J. and Hayward, R. B. “Scalable parallel DFPN search”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2014, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 138-150, doi: 10.1007/978-3-319-09165-5_12.
103. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., . . . Duchesnay, É., “Scikit-learn: Machine learning in Python”, Journal of Machine Learning Research, 12(Oct), pp. 2825-2830, 2011.
104. Pepels, T., Tak, M. J. W., Lanctot, M., and Winands, M. H. M. “Quality-based rewards for Monte-Carlo tree search simulations”, in Proceedings of The 21st European Conference on Artificial Intelligence (ECAI 2014), Prague, Czech Republic, Aug. 18-22, 2014, Schaub, T., Friedrich, G., and O'Sullivan, B. (Eds.), IOS Press, pp. 705-710, doi: 10.3233/978-1-61499-419-0-705.
105. Ponsen, M., Gerritsen, G., and Chaslot, G. M. J. B. “Integrating opponent models with Monte-Carlo tree search in poker”, in Proceedings of AAAI-10 Workshop on Interactive Decision Theory and Game Theory, Atlanta, Georgia, USA, Jul. 12, 2010, AAAI Press, pp. 37-42.
106. Richards, M. and Amir, E. “Opponent modeling in Scrabble”, in Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI-07), Hyderabad, India, 2007, IJCAI Organization, pp. 1482-1487.
107. Robbins, H. and Monro, S., “A stochastic approximation method”, The Annals of Mathematical Statistics, 22(3), pp. 400-407, 1951, doi: 10.1214/aoms/1177729586.
108. Rosin, C. D., “Multi-armed bandits with episode context”, Annals of Mathematics and Artificial Intelligence, 61(3), pp. 203-230, 2011, doi: 10.1007/s10472-011-9258-6.
109. Runarsson, T. P. and Lucas, S. M., “Preference learning for move prediction and evaluation function approximation in Othello”, IEEE Transactions on Computational Intelligence and AI in Games, 6(3), pp. 300-313, 2014, doi: 10.1109/TCIAIG.2014.2307272.
110. Russell, S. and Norvig, P., Artificail Intelligence: A Modern Approach, 3rd edition, Pearson, 1152 pages, 2009.
111. Saffidine, A., Jouandeau, N., Buron, C., and Cazenave, T. “Material symmetry to partition endgame tables”, in Proceedings of The 8th International Conference on Computers and Games (CG 2013), Yokohama, Japan, Aug. 13-15, 2014, van den Herik, H. J., Iida, H., and Plaat, A. (Eds.), Springer International Publishing, pp. 187-198, doi: 10.1007/978-3-319-09165-5_16.
112. Schaeffer, J., Burch, N., Björnsson, Y., Kishimoto, A., Müller, M., Lake, R., . . . Sutphen, S., “Checkers is solved”, Science, 317(5844), pp. 1518-1522, 2007, doi: 10.1126/science.1144079.
113. Schaeffer, J., Hlynka, M., and Jussila, V. “Temporal difference learning applied to a high-performance game-playing program”, in Proceedings of The 17th International Joint Conference on Artificial Intelligence (IJCAI-01), Seattle, Washington, USA, Aug. 4-10, 2001, IJCAI Organization, pp. 529-534.
114. Schaeffer, J. and van den Herik, H. J., “Games, computers, and artificial intelligence”, Artificial Intelligence, 134(1-2), pp. 1-7, 2002, doi: 10.1016/S0004-3702(01)00165-5.
115. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. “Grad-CAM: Visual explanations from deep networks via gradient-based localization”, in Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV 2017), Venice, Italy, Oct. 22-29, 2017, IEEE, pp. 618-626, doi: 10.1109/ICCV.2017.74.
116. Sephton, N., Cowling, P. I., and Slaven, N. H. “An experimental study of action selection mechanisms to create an entertaining opponent”, in Proceedings of 2015 IEEE Conference on Computational Intelligence and Games (CIG 2015), Tainan, Taiwan, Aug. 31-Sep. 2, 2015, IEEE, pp. 122-129, doi: 10.1109/CIG.2015.7317939.
117. Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and de Freitas, N., “Taking the human out of the loop: A review of Bayesian optimization”, Proceedings of the IEEE, 104(1), pp. 148-175, 2016, doi: 10.1109/JPROC.2015.2494218.
118. Shapley, L. S., “Stochastic games”, Proceedings of the National Academy of Sciences, 39(10), pp. 1095-1100, 1953, doi: 10.1073/pnas.39.10.1095.
119. Silva, M. P., Silva, V. d. N., and Chaimowicz, L. “Dynamic difficulty adjustment through an adaptive AI”, in Proceedings of The 14th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames 2015), Teresina, Piauí, Brazil, Nov. 11-13, 2015, Rodrigues, M. A. F., de Carvalho, F. G., and de Vasconcellos, M. S. (Eds.), IEEE, pp. 173-182, doi: 10.1109/SBGames.2015.16.
120. Silver, D., “Reinforcement Learning and Simulation-Based Search in Computer Go”, Department of Computing Science, University of Alberta, Ph.D. dissertation, 158 pages, 2009, Edmonton, Alberta, Canada.
121. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., . . . Hassabis, D., “Mastering the game of Go with deep neural networks and tree search”, Nature, 529(7587), pp. 484-489, 2016, doi: 10.1038/nature16961.
122. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., . . . Hassabis, D., “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play”, Science, 352(6419), pp. 1140-1144, 2018, doi: 10.1126/science.aar6404.
123. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., . . . Hassabis, D., “Mastering the game of Go without human knowledge”, Nature, 550(7676), pp. 354-359, 2017, doi: 10.1038/nature24270.
124. Sirvo, The Rip-offs & Making Our Original Game, retrieved Mar. 5, 2019, from: http://asherv.com/threes/threemails/.
125. Sirvo, THREES-A tiny puzzle that grows on you, retrieved Mar. 5, 2019, from: http://asherv.com/threes/.
126. Sturtevant, N. R. “An analysis of UCT in multi-player games”, in Proceedings of The 6th International Conference on Computers and Games (CG 2008), Beijing, China, Sep. 29-Oct.1, 2008, van den Herik, H. J., Xu, X., Ma, Z., and Winands, M. H. M. (Eds.), Springer Berlin Heidelberg, pp. 37-49, doi: 10.1007/978-3-540-87608-3_4.
127. Su, T.-C., Yen, S.-J., Chen, J.-C., and Wu, I.-C., “TAAI 2012 computer game tournaments”, ICGA Journal, 37(1), pp. 33-35, 2014, doi: 10.3233/ICG-2014-37107.
128. Sun, H., Shao, F., and Li, S. “Design and implementation of the EinStein Würfelt Nicht system based on multi attributes evaluation”, in Proceedings of The 2015 International Conference on Electromechanical Control Technology and Transportation (ICECTT 2015), Zhuhai City, Guangdong Province, China, Oct. 31-Nov. 1, 2015, Atlantis Press, pp. 324-331, doi: 10.2991/icectt-15.2015.62.
129. Sutton, R. S., “Learning to predict by the methods of temporal differences”, Machine Learning, 3(1), pp. 9-44, 1988, doi: 10.1007/BF00115009.
130. Sutton, R. S. and Barto, A. G., Reinforcement Learning: An Introduction, 2nd edition, MIT Press, Cambridge, Massachusetts, USA, 552 pages, 2018.
131. Szubert, M. G. and Jaśkowski, W. “Temporal difference learning of N-tuple networks for the game 2048”, in Proceedings of 2014 IEEE Conference on Computational Intelligence and Games (CIG 2014), Dortmund, Germany, Aug. 26-29, 2014, IEEE, doi: 10.1109/CIG.2014.6932907.
132. Takeuchi, S., Kaneko, T., and Yamaguchi, K., “Evaluation of game tree search methods by game records”, IEEE Transactions on Computational Intelligence and AI in Games, 2(4), pp. 288-302, 2010, doi: 10.1109/TCIAIG.2010.2102022.
133. Tensorflow, tensorflow/minigo: An open-source implementation of the AlphaGoZero algorithm, retrieved Mar. 5, 2019, from: https://github.com/tensorflow/minigo.
134. Tesauro, G., “Temporal difference learning and TD-Gammon”, Communications of the ACM, 38(3), pp. 58-68, 1995, doi: 10.1145/203330.203343.
135. Thill, M., Bagheri, S., Koch, P., and Konen, W. “Temporal difference learning with eligibility traces for the game connect four”, in Proceedings of 2014 IEEE Conference on Computational Intelligence and Games, Dortmund, Germany, Aug. 26-29, 2014, IEEE, doi: 10.1109/CIG.2014.6932870.
136. Thill, M., Koch, P., and Konen, W. “Reinforcement learning with n-tuples on the game Connect-4”, in Proceedings of The 12th International Conference on Parallel Problem Solving from Nature (PPSN 2012), Taormina, Italy, Sep. 1-5, 2012, Coello, C. A. C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., and Pavone, M. (Eds.), Springer Berlin Heidelberg, pp. 184-194, doi: 10.1007/978-3-642-32937-1_19.
137. Thompson, K., “Retrograde analysis of certain endgames”, ICCA Journal, 9(3), pp. 131-139, 1986.
138. Tian, Y. and Zhu, Y. “Better computer Go player with neural network and long-term prediction”, in The 4th International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico, May 2-4, May 2-4, 2016 of Conference.
139. Trinh, T. B., Bashi, A. S., and Deshpande, N. “Temporal difference learning in Chinese Chess”, in Proceedings of The 11th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems (IEA-98-AIE), Benicàssim, Castellón, Spain, Jun. 1-4, 1998, del Pobil, A. P., Mira, J., and Ali, M. (Eds.), Springer Berlin Heidelberg, pp. 612-618, doi: 10.1007/3-540-64574-8_447.
140. Tseng, W.-J., Chen, J.-C., Chen, L.-P., Yen, S.-J., and Wu, I.-C., “TCGA 2013 computer game tournaments”, ICGA Journal, 36(3), pp. 166-168, 2013, doi: 10.3233/ICG-2013-36310.
141. Tseng, W.-J., Chen, J.-C., and Wu, I.-C., “DarkKnight wins Chinese dark chess tournament”, ICGA Journal, 39(2), pp. 163-165, 2017, doi: 10.3233/ICG-170023.
142. van den Dries, S. and Wiering, M. A., “Neural-fitted TD-leaf learning for playing Othello with structured neural networks”, IEEE Transactions on Neural Networks and Learning Systems, 23(11), pp. 1701-1713, 2012, doi: 10.1109/TNNLS.2012.2210559.
143. van den Herik, H. J., Iida, H., Plaat, A., and Hellemons, J., “The brain and mind sports Olympiad”, ICGA Journal, 36(3), pp. 172, 2013, doi: 10.3233/ICG-2013-36313.
144. van den Herik, H. J., Plaat, A., and Hellemons, J., “The 16th Computer Olympiad”, ICGA Journal, 35(1), pp. 50, 2012, doi: 10.3233/ICG-2012-35111.
145. van den Herik, H. J., Uiterwijk, J. W. H. M., and van Rijswijck, J., “Games solved: Now and in the future”, Artificial Intelligence, 134(1-2), pp. 277-311, 2002, doi: 10.1016/S0004-3702(01)00152-7.
146. Van Lishout, F., Chaslot, G. M. J. B., and Uiterwijk, J. W. H. M. “Monte-Carlo tree search in backgammon”, in Proceedings of Computer Games Workshop 2007 (CGW 2007), Amsterdam, the Netherlands, Jun. 15-17, 2007, van den Herik, H. J., Uiterwijk, J. W. H. M., Winands, M. H. M., and Schadd, M. P. D. (Eds.), pp. 175-184.
147. Veness, J., Silver, D., Uther, W., and Blair, A. “Bootstrapping from game tree search”, in Proceedings of The 22nd International Conference on Neural Information Processing Systems (NIPS 2009), Vancouver, British Columbia, Canada, 2009, Bengio, Y., Schuurmans, D., Williams, C. K. I., and Culotta, A. (Eds.), NIPS Foundation, Inc., pp. 1937-1945.
148. Wang, J., Zhu, T., Li, H., Hsueh, C.-H., and Wu, I.-C. “Belief-state Monte-Carlo tree search for phantom games”, in Proceedings of 2015 IEEE Conference on Computational Intelligence and Games (CIG 2015), Tainan, Taiwan, Aug. 31-Sep. 2, 2015, IEEE, pp. 267-274, doi: 10.1109/CIG.2015.7317917.
149. Wang, J., Zhu, T., Li, H., Hsueh, C.-H., and Wu, I.-C., “Belief-state Monte Carlo tree search for phantom Go”, IEEE Transactions on Games, 10(2), pp. 139-154, 2018, doi: 10.1109/TCIAIG.2017.2734067.
150. Wikipedia, Tabletop game - Wikipedia, retrieved Mar. 5, 2019, from: https://en.wikipedia.org/wiki/Tabletop_game.
151. Winands, M. H. M., Björnsson, Y., and Saito, J.-T. “Monte-Carlo tree search solver”, in Proceedings of The 6th International Conference on Computers and Games (CG 2008), Beijing, China, Sep. 29-Oct. 1, 2008, van den Herik, H. J., Xu, X., Ma, Z., and Winands, M. H. M. (Eds.), Springer Berlin Heidelberg, pp. 25-36, doi: 10.1007/978-3-540-87608-3_3.
152. Winands, M. H. M., Björnsson, Y., and Saito, J.-T., “Monte Carlo tree search in lines of action”, IEEE Transactions on Computational Intelligence and AI in Games, 2(4), pp. 239-250, 2010, doi: 10.1109/TCIAIG.2010.2061050.
153. Wu, I.-C., Tsai, H.-T., Lin, H.-H., Lin, Y.-S., Chang, C.-M., and Lin, P.-H. “Temporal difference learning for Connect6”, in Proceedings of The 13th International Conference on Advances in Computer Games (ACG 2011), Tilburg, the Netherlands, Nov. 20-22, 2011, van den Herik, H. J. and Plaat, A. (Eds.), Springer Berlin Heidelberg, pp. 121-133, doi: 10.1007/978-3-642-31866-5_11.
154. Wu, I.-C., Wu, T.-R., Liu, A.-J., Guei, H., and Wei, T. h. “On strength adjustment for MCTS-based programs”, in Proceedings of The 33rd AAAI Conference on Artificial Intelligence (AAAI-19), Honolulu, Hawaii, USA, Jan. 27-Feb. 1, 2019, AAAI Press.
155. Wu, I.-C., Yeh, K.-H., Liang, C.-C., Chang, C.-C., and Chiang, H. “Multi-stage temporal difference learning for 2048”, in Proceedings of 2014 Conference on Technologies and Applications of Artificial Intelligence (TAAI 2014), Taipei, Taiwan, Nov. 21-23, 2014, Cheng, S.-M. and Day, M.-Y. (Eds.), Springer International Publishing, pp. 366-378, doi: 10.1007/978-3-319-13987-6_34.
156. Wu, T.-R., Wu, I.-C., Chen, G.-W., Wei, T. h., Wu, H.-C., Lai, T.-Y., and Lan, L.-C., “Multilabeled value networks for computer Go”, IEEE Transactions on Games, 10(4), pp. 378-389, 2018, doi: 10.1109/TG.2018.2852806.
157. Yang, J.-K., Su, T.-C., and Wu, I.-C., “TCGA 2012 computer games tournament”, ICGA Journal, 35(3), pp. 178-180, 2012, doi: 10.3233/ICG-2012-35305.
158. Yannakakis, G. N. and Hallam, J. “Evolving opponents for interesting interactive computer games”, in Proceedings of The 8th International Conference on the Simulation of Adaptive Behavior, From Animals to Animats 8 (SAB 2004), Los Angeles, California, USA, Jul. 13-17, 2004, Schaal, S., Ijspeert, A. J., Billard, A., Vijayakumar, S., Hallam, J., and Meyer, J.-A. (Eds.), MIT Press, pp. 499-508.
159. Yeh, K.-H., Wu, I.-C., Hsueh, C.-H., Chang, C.-C., Liang, C.-C., and Chiang, H., “Multistage temporal difference learning for 2048-like games”, IEEE Transactions on Computational Intelligence and AI in Games, 9(4), pp. 369-380, 2017, doi: 10.1109/TCIAIG.2016.2593710.
160. Yen, S.-J., Chen, J.-C., Chen, B.-N., and Tseng, W.-J., “DarkKnight wins Chinese dark chess tournament”, ICGA Journal, 36(3), pp. 175-176, 2013, doi: 10.3233/ICG-2013-36315.
161. Yen, S.-J., Chiu, S.-Y., and Wu, I.-C., “MoDark wins the Chinese dark chess tournament”, ICGA Journal, 33(4), pp. 230-231, 2010, doi: 10.3233/ICG-2010-33410.
162. Yen, S.-J., Chou, C.-W., Chen, J.-C., Wu, I.-C., and Kao, K.-Y., “Design and implementation of Chinese dark chess programs”, IEEE Transactions on Computational Intelligence and AI in Games, 7(1), pp. 66-74, 2015, doi: 10.1109/TCIAIG.2014.2329034.
163. Yen, S.-J., Su, T.-C., and Wu, I.-C., “The TCGA 2011 computer-games tournament”, ICGA Journal, 34(2), pp. 108-110, 2011, doi: 10.3233/ICG-2011-34216.
164. Yin, H.-F. and Fu, T.-T. “Applying temporal difference learning to acquire a high-performance position evaluation function”, in Proceedings of The 7th International Conference on Computer Science & Education (ICCSE 2012), Melbourne, Australia, Jul. 14-17, 2012, IEEE, pp. 80-84, doi: 10.1109/ICCSE.2012.6295031.
165. Zohaib, M., “Dynamic difficulty adjustment (DDA) in computer games: A review”, Advances in Human-Computer Interaction, 2018, pp. 1-12, 2018, doi: 10.1155/2018/5681652.