臺灣博碩士論文加值系統

摘要 i
Abstract iii
Content vii
List of Figures xi
List of Tables xiv
List of Abbreviations xv
Chapter 1 Introduction 1
1.1 Research Background and Motivation 1
1.2 Literature Review 3
1.2.1 Traditional Neural Networks 3
1.2.2 Deep Learning Methodologies 7
1.3 The Contribution of This Study 12
1.4 Dissertation Structure 15
Chapter 2 Introduction to Distribution Insulator 18
2.1 Distribution Feeder System in Taiwan 18
2.2 Distribution Insulator Characteristic 21
2.3 Insulator leakage Current Characteristic 28
Chapter 3 The Online Monitoring System 32
3.1 The Architecture of Online Monitoring System 32
3.2 The Data Collection System 35
3.2.1 The Weather Parameter Collecting System 35
3.2.2 The Leakage Current Collecting Device 39
3.2.3 The Insulator Image Collecting Module 44
3.3 The Back-end Monitoring Platform System 46
3.4 Processing The Sequential Data 54
Chapter 4 Traditional Methodology- Particle Swarm Optimization Based Neural Network 55
4.1 Collecting Data 55
4.2 Enhancement Inputs 59
4.3 Input Data Model 63
4.4 Persistent Predicting Models 64
4.5 Particle Swarm Optimization Combined Back Propagation Neural Network 69
4.6 Error metric 75
Chapter 5 Deep Learning Algorithm: Convolution NN Bidirectional Long Short-Term Memory 77
5.1 Collecting Weather Parameters and Leakage Current Levels 77
5.2 The Long Short-Term Memory and Gated Recurrent Unit 80
5.3 The Bidirectional Long Short-Term Memory 83
5.4 The Proposed Convolutional Neural Network and the Combination with the Bidirectional Long Short-Term Memory 84
5.5 Evaluating Metrics 86
5.6 The Selection of Input Parameters 87
Chapter 6 Experiment Results for Traditional Neural Network and Deep Learning Methodologies 89
6.1 Traditional Neural Network 89
6.2 Deep Learning Methodology 97
6.2.1 Parameter Optimization for CNN-Bi-LSTM 97
6.2.2 Evaluating the Performance of CNN-Bi-LSTM 103
6.2.3 Results and Findings 111
6.3 Discussion 113
6.3.1 Traditional Neural Network 113
6.3.2 Discussion for CNN-Bi-LSTM Algorithm 114
Chapter 7 The Integration of Deep Learning AI in the Web-Based Service 116
7.1 Introduction to the Deep Learning TensorFlow AI Library 116
7.2 Integrated the TensorFlow on the Web-based Service 120
7.3 Hyperparameter Optimization Process in Deep Learning Machine 123
7.3.1 Preparing the Collected Data in the Excel File 125
7.3.2 Selecting the Appropriate Input Features Based on Pearson Correlation Matrix 126
7.3.3 Choosing the Configuration for Hyperparameter Optimization 129
7.3.4 Comparing the Performance Between Configurations 130
7.4 Comparing the Performance Between Different Models 133
7.4.1 Getting the Optimized Configuration from the Hyperparameter Process 133
7.4.2 Choosing Different Models for Comparison 135
7.4.3 Comparing the Performance Between Deep Learning Methodologies 137
7.4.4 Exporting the Training Model for Prediction and Transfer Learning Technology 137
7.5 Transfer Learning Technique 139
Chapter 8 Conclusion and Future Research 143
8.1 Conclusion 143
8.2 Future Research 144
Reference 146
Publication List 160
Autobiography 164

[1]M. A.Salam, S. P.Ang, B. T.Ong, O. A.Malik, W.Voon, andM.Alinurrezan, “Measurement of pollution level of 66 kV transmission line insulators,” in 2013 Annual Report Conference on Electrical Insulation and Dielectric Phenomena, Oct. 2013, pp. 1124–1127, doi: 10.1109/CEIDP.2013.6748144.
[2]C.Gu, G.Lu, M.Yi, andJ.Li, “Study on artificial contamination test of typical transmission line insulators,” in Proceedings of the 5th IEEE International Conference on Electric Utility Deregulation, Restructuring and Power Technologies, DRPT 2015, Nov. 2016, pp. 1624–1627, doi: 10.1109/DRPT.2015.7432502.
[3]X.Lu, X.Chen, D.Zhang, andZ.Zhang, “Analysis on natural contamination depositing characteristics of imitative insulator string for overhead transmission line in Shantou,” in Annual Report - Conference on Electrical Insulation and Dielectric Phenomena, CEIDP, Oct. 2013, pp. 452–455, doi: 10.1109/CEIDP.2013.6748137.
[4]G.Montoya-Tena, R.Hernández-Corona, andI.Ramírez-Vázquez, “Experiences on pollution level measurement in Mexico,” Electr. Power Syst. Res., 2005, doi: 10.1016/j.epsr.2005.04.003.
[5]M. A.Abouelsaad, M. A.Abouelatta, B.Arafa, andM. E.Ibrahim, “Environmental pollution effects on insulators of northern Egypt HV transmission lines,” 2013, doi: 10.1109/CEIDP.2013.6748153.
[6]Y.-C.Wang, Y.-T.Lin, H.-C.Chang, andC.-C.Kuo, “Contamination assessment of insulators using microsystem technology with fuzzy-based approach,” Microsyst. Technol., pp. 1–14, 2019.
[7]A.Banik, A. K.Pradhan, R.Ghosh, S.Dalai, andB.Chaterjee, “A comparative study on leakage current harmonics of porcelain disc insulator contaminated with NaCl and KCl,” in 2015 1st Conference on Power, Dielectric and Energy Management at NERIST (ICPDEN), Jan. 2015, pp. 1–4, doi: 10.1109/ICPDEN.2015.7084496.
[8]G.Ramos N., M. T.Campillo R., andK.Naito, “A study on the characteristics of various conductive contaminants accumulated on high voltage insulators,” IEEE Trans. Power Deliv., vol. 8, no. 4, pp. 1842–1850, Oct.1993, doi: 10.1109/61.248293.
[9]A. S.Sidthik, L.Kalaivani, andM. W.Iruthayarajan, “Evaluation and prediction of contamination level in coastal region insulators based on leakage current characteristics,” in 2013 International Conference on Circuits, Power and Computing Technologies (ICCPCT), Mar. 2013, pp. 132–137, doi: 10.1109/ICCPCT.2013.6528878.
[10]Jeong-Ho Kim et al., “Leakage current monitoring and outdoor degradation of silicone rubber,” IEEE Trans. Dielectr. Electr. Insul., vol. 8, no. 6, pp. 1108–1115, Dec.2001, doi: 10.1109/94.971471.
[11]S.Deb, R.Ghosh, S.Dutta, S.Dalai, andB.Chatterjee, “Effect of humidity on leakage current of a contaminated 11 kV Porcelain Pin Insulator,” in 2017 6th International Conference on Computer Applications in Electrical Engineering - Recent Advances, CERA 2017, 2018, vol. 2018-Janua, pp. 215–219, doi: 10.1109/CERA.2017.8343329.
[12]G.Zhicheng, M.Yingke, W.Liming, L.Ruihai, W.Hua, andM.Yi, “Leakage current and discharge phenomenon of outdoor insulators,” Int. J. Electr. Eng. Informatics, vol. 1, no. 1, p. 1, 2009, doi: 10.15676/ijeei.2009.1.1.1.
[13]S.Anjum, A.El-Hag, S.Jayaram, andA.Naderian, “Classification of defects in ceramic insulators using partial discharge signatures extracted from radio frequency (RF) signals,” in 2014 IEEE Conference on Electrical Insulation and Dielectric Phenomena, CEIDP 2014, 2014, pp. 212–215, doi: 10.1109/CEIDP.2014.6995770.
[14]M. M.Hussain, S.Farokhi, S. G.McMeekin, andM.Farzaneh, “Prediction of surface degradation of composite insulators using PD measurement in cold fog,” in Proceedings of the 2016 IEEE International Conference on Dielectrics, ICD 2016, 2016, vol. 2, pp. 697–700, doi: 10.1109/ICD.2016.7547711.
[15]R.Salustiano et al., “Development of new methodology for insulators inspections on aerial distribution lines based on partial discharge detection tools,” 2014, doi: 10.1109/ICHVE.2014.7035429.
[16]H.Ali, “Leakage current prediction for high voltage insulators flashover based on extreme value theory,” in Proceedings - 2016 IEEE International Symposium on Computer, Consumer and Control, IS3C 2016, 2016, pp. 870–873, doi: 10.1109/IS3C.2016.221.
[17]L.Wang, X.Li, B.Cao, andS.Gao, “Prediction of leakage current on insulator surface of transmission line based on BP neural network,” in Gaoya Dianqi/High Voltage Apparatus, 2020, vol. 56, no. 2, pp. 69–76, doi: 10.13296/j.1001-1609.hva.2020.02.011.
[18]C.Volat, F.Meghnefi, M.Farzaneh, andH.Ezzaidi, “Monitoring leakage current of ice-covered station post insulators using artificial neural networks,” IEEE Trans. Dielectr. Electr. Insul., vol. 17, no. 2, pp. 443–450, 2010, doi: 10.1109/TDEI.2010.5448099.
[19]A.Naderian Jahromi, A. H.El-Hag, E. A.Cherney, S. H.Jayaram, M.Sanaye-Pasand, andH.Mohseni, “Prediction of leakage current of composite insulators in salt fog test using neural network,” in Annual Report - Conference on Electrical Insulation and Dielectric Phenomena, CEIDP, 2005, vol. 2005, pp. 309–312, doi: 10.1109/CEIDP.2005.1560683.
[20]A. N.Jahromi, A. H.El-Hag, S. H.Jayaram, E. A.Cherney, M.Sanaye-Pasand, andH.Mohseni, “A neural network based method for leakage current prediction of polymeric insulators,” IEEE Trans. Power Deliv., vol. 21, no. 1, pp. 506–507, 2006, doi: 10.1109/TPWRD.2005.858805.
[21]Y. K.Mao, Z. C.Guan, L. M.Wang, andB.Yue, “Prediction of leakage current of outdoor insulators based on BP artificial neural network,” in Zhongguo Dianji Gongcheng Xuebao/Proceedings of the Chinese Society of Electrical Engineering, 2007, vol. 27, no. 27, pp. 7–12.
[22]A. H.El-Hag, A. N.Jahromi, andM.Sanaye-Pasand, “Prediction of leakage current of non-ceramic insulators in early aging period,” Electr. Power Syst. Res., vol. 78, no. 10, pp. 1686–1692, 2008, doi: 10.1016/j.epsr.2008.02.010.
[23]J. L.Fierro-Chavez, Z.Ramirez-Vazquez, andG.Montoya-Tena, “On-line leakage current monitoring of 400 kV insulator strings in polluted areas,” IEE Proceedings-Generation, Transm. Distrib., vol. 143, no. 6, pp. 560–564, 1996.
[24]A. H.El-Hag, S. H.Jayaram, andE. A.Cherney, “Fundamental and low frequency harmonic components of leakage current as a diagnostic tool to study aging of RTV and HTV silicone rubber in salt-fog,” IEEE Trans. Dielectr. Electr. Insul., vol. 10, no. 1, pp. 128–136, 2003.
[25]D.Devendranath andA. D.Rajkumar, “Leakage current and charge in RTV coated insulators under pollution conditions,” IEEE Trans. Dielectr. Electr. Insul., vol. 9, no. 2, pp. 294–299, 2002.
[26]A. K.Chaou, A.Mekhaldi, B.Moula, andM.Teguar, “Classification of Leakage Current waveforms using Wavelet Packet Transform on high voltage insulator,” in 2014 ICHVE International Conference on High Voltage Engineering and Application, 2014, pp. 1–4.
[27]P.Govindaraju andC.Muniraj, “Monitoring and optimizing the state of pollution of high voltage insulators using wireless sensor network based convolutional neural network,” Microprocess. Microsyst., vol. 79, 2020.
[28]G.Montoya, I.Ramirez, andR.Hernandez, “The leakage current as a diagnostic tool for outdoor insulation,” in 2008 IEEE/PES Transmission and Distribution Conference and Exposition: Latin America, 2008, pp. 1–4.
[29]A. N.Jahromi, A. H.El-Hag, S. H.Jayaram, E. A.Cherney, M.Sanaye-Pasand, andH.Mohseni, “A neural network based method for leakage current prediction of polymeric insulators,” IEEE Trans. Power Deliv., vol. 21, no. 1, pp. 506–507, 2005.
[30]H.deSantos andM. Á.Sanz-Bobi, “A Cumulative Pollution Index for the Estimation of the Leakage Current on Insulator Strings,” IEEE Trans. Power Deliv., vol. 35, no. 5, pp. 2438–2446, 2020, doi: 10.1109/TPWRD.2020.2968556.
[31]M.Otsubo, T.Hashiguchi, C.Honda, O.Takenouchi, T.Sakoda, andY.Hashimoto, “Evaluation of insulation performance of polymeric surface using a novel separation technique of leakage current,” IEEE Trans. Dielectr. Electr. Insul., vol. 10, no. 6, pp. 1053–1060, 2003.
[32]E.Thalassinakis andC. G.Karagiannopoulos, “Measurements and interpretations concerning leakage currents on polluted high voltage insulators,” Meas. Sci. Technol., vol. 14, no. 4, p. 421, 2003.
[33]F.Amarh, Electric transmission line flashover prediction system. Arizona state university, 2001.
[34]M. A. R. M.Fernando, Performance of non-ceramic insulators in tropical environments. Chalmers University of Technology, 1999.
[35]S.Gao et al., “Prediction method of leakage current of insulators on the transmission line based on BP neural network,” in 2018 IEEE 2nd International Electrical and Energy Conference (CIEEC), Nov. 2018, pp. 569–572, doi: 10.1109/CIEEC.2018.8745839.
[36]W. L.Vosloo andJ. P.Holtzhausen, “The prediction of insulator leakage currents from environmental data,” in IEEE AFRICON. 6th Africon Conference in Africa, 2002, vol. 2, pp. 603–608 vol.2, doi: 10.1109/AFRCON.2002.1159978.
[37]A. S.Ahmad, P. S.Ghosh, S. A. K.Aljunid, andH.Ahmad, “Modeling of various meteorological effects on contamination level for suspension type of high voltage insulators using ANN,” in IEEE/PES Transmission and Distribution Conference and Exhibition, 2002, vol. 2, pp. 1030–1035 vol.2, doi: 10.1109/TDC.2002.1177619.
[38]C.Zachariades, S. M.Rowland, andI.Cotton, “Real-time monitoring of leakage current on insulating cross-arms in relation to local weather conditions,” in 2013 IEEE Electrical Insulation Conference (EIC), 2013, pp. 397–401, doi: 10.1109/EIC.2013.6554275.
[39]M. A.Pinotti andL. H.Meyer, “Mathematical model for prediction of the leakage current on distribution insulators of 25 kV class,” in 2017 IEEE Electrical Insulation Conference (EIC), 2017, pp. 256–260, doi: 10.1109/EIC.2017.8004689.
[40]S. M.Elkhodary andL. S.Nasrat, “The use of experimental and artificial neural network technique to estimate age against surface leakage current for non-ceramic insulators,” in 2006 Large Engineering Systems Conference on Power Engineering, 2006, pp. 84–89.
[41]P.Cline, W.Lannes, andG.Richards, “Use of pollution monitors with a neural network to predict insulator flashover,” Electr. Power Syst. Res., vol. 42, no. 1, pp. 27–33, 1997.
[42]S.Kumagai andN.Yoshimura, “Leakage current characterization for estimating the conditions of ceramic and polymeric insulating surfaces,” IEEE Trans. Dielectr. Electr. Insul., vol. 11, no. 4, pp. 681–690, 2004.
[43]L.Zhao et al., “The prediction of post insulators leakage current from environmental data,” in 2011 International Conference on Electrical and Control Engineering, 2011, pp. 5103–5106, doi: 10.1109/ICECENG.2011.6057235.
[44]N.AlKhafaf andA. H.El-Hag, “Prediction of leakage current peak value,” in 2018 11th International Symposium on Mechatronics and its Applications (ISMA), 2018, pp. 1–4.
[45]P. N.Thanh, M.-Y.Cho, andT. N.Da, “Insulator leakage current prediction using surface spark discharge data and particle swarm optimization based neural network,” Electr. Power Syst. Res., vol. 191, p. 106888.
[46]Y.LeCun, Y.Bengio, andG.Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
[47]J. J.Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci., vol. 79, no. 8, pp. 2554–2558, 1982.
[48]H.Liu, C.Chen, X.Lv, X.Wu, andM.Liu, “Deterministic wind energy forecasting: A review of intelligent predictors and auxiliary methods,” Energy Convers. Manag., vol. 195, pp. 328–345, 2019.
[49]H. A.Kazem andJ. H.Yousif, “Comparison of prediction methods of photovoltaic power system production using a measured dataset,” Energy Convers. Manag., vol. 148, pp. 1070–1081, 2017.
[50]O.Laib, M. T.Khadir, andL.Mihaylova, “Toward efficient energy systems based on natural gas consumption prediction with LSTM Recurrent Neural Networks,” Energy, vol. 177, pp. 530–542, 2019.
[51]S.Srivastava andS.Lessmann, “A comparative study of LSTM neural networks in forecasting day-ahead global horizontal irradiance with satellite data,” Sol. Energy, vol. 162, pp. 232–247, 2018.
[52]Z.Zhang et al., “Long short-term memory network based on neighborhood gates for processing complex causality in wind speed prediction,” Energy Convers. Manag., vol. 192, pp. 37–51, 2019.
[53]W.Kong, Z. Y.Dong, Y.Jia, D. J.Hill, Y.Xu, andY.Zhang, “Short-term residential load forecasting based on LSTM recurrent neural network,” IEEE Trans. Smart Grid, vol. 10, no. 1, pp. 841–851, 2017.
[54]S.Atef andA. B.Eltawil, “Assessment of stacked unidirectional and bidirectional long short-term memory networks for electricity load forecasting,” Electr. Power Syst. Res., vol. 187, p. 106489, 2020.
[55]S.Wang, X.Wang, S.Wang, andD.Wang, “Bi-directional long short-term memory method based on attention mechanism and rolling update for short-term load forecasting,” Int. J. Electr. Power Energy Syst., vol. 109, pp. 470–479, 2019.
[56]T.Le, M. T.Vo, B.Vo, E.Hwang, S.Rho, andS. W.Baik, “Improving electric energy consumption prediction using CNN and Bi-LSTM,” Appl. Sci., vol. 9, no. 20, p. 4237, 2019.
[57]F. R.Alharbi andD.Csala, “Short-Term Solar Irradiance Forecasting Model Based on Bidirectional Long Short-Term Memory Deep Learning,” in 2021 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), 2021, pp. 1–6.
[58]F. U. M.Ullah, A.Ullah, I. U.Haq, S.Rho, andS. W.Baik, “Short-term prediction of residential power energy consumption via CNN and multi-layer bi-directional LSTM networks,” IEEE Access, vol. 8, pp. 123369–123380, 2019.
[59]S.Munawar, M.Asif, B.Kabir, A.Ullah, andN.Javaid, “Electricity Theft Detection in Smart Meters Using a Hybrid Bi-directional GRU Bi-directional LSTM Model,” in Conference on Complex, Intelligent, and Software Intensive Systems, 2021, pp. 297–308.
[60]T.-Y.Kim andS.-B.Cho, “Predicting residential energy consumption using CNN-LSTM neural networks,” Energy, vol. 182, pp. 72–81, 2019.
[61]C. C.Aggarwal, “Neural networks and deep learning,” Springer, vol. 10, pp. 973–978, 2018.
[62]J.Schmidhuber, “Deep learning in neural networks: An overview,” Neural networks, vol. 61, pp. 85–117, 2015.
[63]M. A.Nielsen, Neural networks and deep learning, vol. 25. Determination press San Francisco, CA, USA, 2015.
[64]T.Company, “Taiwan Power Company 2020 Sustainability Report,” 2020. [Online]. Available: https://www.taipower.com.tw/upload/4461/2020111211185619027.pdf.
[65]B. K.Bose, “Neural Network and Applications,” Power Electron. Mot. Drives, vol. 65, pp. 731–850, 2006, doi: 10.1016/b978-012088405-6/50013-3.
[66]T.Tchaban, J. P.Griffin, andM. J.Taylor, “A comparison between single and combined backpropagation neural networks in the prediction of turnover,” Eng. Appl. Artif. Intell., vol. 11, no. 1, pp. 41–47, 1998, doi: 10.1016/S0952-1976(97)00059-6.
[67]F.Schwenker, H. A.Kestler, andG.Palm, Three learning phases for radial-basis-function networks, vol. 14, no. 4–5. Neural Networks, 2001.
[68]M.Ester, A.Frommelt, andH.Kriegel, “Algorithms for Characterization and Trend Detection in Spatial Databases,” in KDD-98 Proceedings, 1998, pp. 44–50.
[69]D. E.Rumelhart andD.Zipser, Feature discovery by competitive learning, vol. 9, no. 1. Cognitive Science, 1985.
[70]S. R.Sain andV. N.Vapnik, The Nature of Statistical Learning Theory, vol. 38, no. 4. New York: Springer, 1996.
[71]C. C.Chang andC. J.Lin, “LIBSVM: A Library for support vector machines,” ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, 2011, doi: 10.1145/1961189.1961199.
[72]A. J.Smola andB.Schölkopf, A tutorial on support vector regression, vol. 14, no. 3. Statistics and Computing, 2004.
[73]C.Huang, L. S.Davis, andJ. R. G.Townshend, “An assessment of support vector machines for land cover classification,” Int. J. Remote Sens., vol. 23, no. 4, pp. 725–749, 2002, doi: 10.1080/01431160110040323.
[74]M.Clerc, “Particle Swarm Optimization,” 2010, doi: 10.1002/9780470612163.
[75]V. G.Gudise andG. K.Venayagamoorthy, “Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks,” in 2003 IEEE Swarm Intelligence Symposium, SIS 2003 - Proceedings, 2003, pp. 110–117, doi: 10.1109/SIS.2003.1202255.
[76]X.Hu, R. C.Eberhart, andY.Shi, “Engineering optimization with particle swarm,” in 2003 IEEE Swarm Intelligence Symposium, SIS 2003 - Proceedings, 2003, pp. 53–57, doi: 10.1109/SIS.2003.1202247.
[77]M. A.Panduro, C. A.Brizuela, L. I.Balderas, andD. A.Acosta, “A comparison of genetic algorithms, particle swarm optimization and the differential evolution method for the design of scannable circular antenna arrays,” Prog. Electromagn. Res. B, no. 13, pp. 171–186, 2009, doi: 10.2528/PIERB09011308.
[78]S.Panda andN. P.Padhy, “Comparison of particle swarm optimization and genetic algorithm for FACTS-based controller design,” Appl. Soft Comput. J., vol. 8, no. 4, pp. 1418–1427, 2008, doi: 10.1016/j.asoc.2007.10.009.
[79]Y.Shi andR.Eberhart, “Modified particle swarm optimizer,” in Proceedings of the IEEE Conference on Evolutionary Computation, ICEC, 1998, pp. 69–73, doi: 10.1109/icec.1998.699146.
[80]Y.Hua, Z.Zhao, R.Li, X.Chen, Z.Liu, andH.Zhang, “Deep learning with long short-term memory for time series prediction,” IEEE Commun. Mag., vol. 57, no. 6, pp. 114–119, 2019.
[81]S.Hochreiter andJ.Schmidhuber, “Long Short-term Memory,” Neural Comput., vol. 9, pp. 1735–1780, Dec.1997, doi: 10.1162/neco.1997.9.8.1735.
[82]K.Greff, R. K.Srivastava, J.Koutník, B. R.Steunebrink, andJ.Schmidhuber, “LSTM: A search space odyssey,” IEEE Trans. neural networks Learn. Syst., vol. 28, no. 10, pp. 2222–2232, 2016.
[83]W.Zheng andG.Chen, “An Accurate GRU-Based Power Time-Series Prediction Approach With Selective State Updating and Stochastic Optimization,” IEEE Trans. Cybern.
[84]K.Cho et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv Prepr. arXiv1406.1078, 2014.
[85]J.Chung, C.Gulcehre, K.Cho, andY.Bengio, “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,” Dec.2014.
[86]A.Krizhevsky, I.Sutskever, andG. E.Hinton, “Imagenet classification with deep convolutional neural networks,” Adv. Neural Inf. Process. Syst., vol. 25, pp. 1097–1105, 2012.
[87]M. D.Zeiler andR.Fergus, “Visualizing and understanding convolutional networks,” in European conference on computer vision, 2014, pp. 818–833.
[88]J.Gu et al., “Recent advances in convolutional neural networks,” Pattern Recognit., vol. 77, pp. 354–377, 2018.
[89]S.Qian, H.Liu, C.Liu, S.Wu, andH.San Wong, “Adaptive activation functions in convolutional neural networks,” Neurocomputing, vol. 272, pp. 204–212, 2018.
[90]G.Lin andW.Shen, “Research on convolutional neural network based on improved Relu piecewise activation function,” Procedia Comput. Sci., vol. 131, pp. 977–984, 2018.
[91]D. P.Kingma andJ.Ba, “Adam: A method for stochastic optimization,” arXiv Prepr. arXiv1412.6980, 2014.
[92]T.Dozat, “Incorporating nesterov momentum into adam,” 2016.
[93]J. S.Bridle, “Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition,” in Neurocomputing, Springer, 1990, pp. 227–236.
[94]J.Bergstra andY.Bengio, “Random search for hyper-parameter optimization.,” J. Mach. Learn. Res., vol. 13, no. 2, 2012.
[95]Y.Bengio andY.LeCun, “Scaling learning algorithms towards AI,” Large-scale kernel Mach., vol. 34, no. 5, pp. 1–41, 2007.
[96]V.Nair andG. E.Hinton, “Rectified linear units improve restricted boltzmann machines,” 2010.
[97]P.Goldsborough, “A tour of tensorflow,” arXiv Prepr. arXiv1610.01178, 2016.
[98]R.Collobert, S.Bengio, andJ.Mariéthoz, “Torch: a modular machine learning software library,” Idiap, 2002.
[99]F.Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.
[100]R.Al-Rfou et al., “Theano: A Python framework for fast computation of mathematical expressions,” arXiv e-prints, p. arXiv-1605, 2016.
[101]M.Abadi et al., “TensorFlow: A System for Large-Scale Machine Learning,” in 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), Nov. 2016, pp. 265–283, [Online]. Available: https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi.
[102]R.Kohavi, D.Sommerfield, andJ.Dougherty, “Data mining using a machine learning library in C++,” Int. J. Artif. Intell. Tools, vol. 6, no. 04, pp. 537–566, 1997.
[103]G.Bradski, “The openCV library.,” Dr. Dobb’s J. Softw. Tools Prof. Program., vol. 25, no. 11, pp. 120–123, 2000.
[104]A.Bifet et al., “Moa: Massive online analysis, a framework for stream classification and clustering,” in Proceedings of the first workshop on applications of pattern analysis, 2010, pp. 44–50.
[105]Z.Li, F.Liu, W.Yang, S.Peng, andJ.Zhou, “A survey of convolutional neural networks: analysis, applications, and prospects,” IEEE Trans. Neural Networks Learn. Syst., 2021.