臺灣博碩士論文加值系統

隨著空氣汙染成為全球重視的議題，科學家們便致力於空氣汙染的研究。在空氣汙染預報的領域，至今在實驗研究上已有不錯的成效，但鮮少研究考量天氣預報資訊及探討空氣汙染隨風飄散的特性。本研究以基於注意機制之神經網路模型學習風向與風速時間序列對鄰近地區PM2.5濃度隨時空變化的影響力，接著以長短期記憶神經網路進行初步預測，最後我們以基於梯度提升樹之整體學習方式結合PM2.5初步預測與天氣預報資訊，進行二階段PM2.5預測。本研究透過環保署觀測站資料及中央氣象局天氣預報資料進行實驗。而實驗結果顯示本研究提出的方法在預測中長期PM2.5的準確性優於其他預測模型，包含線性回歸、多層感知器、支援向量機、長短期記憶神經網路及梯度提升樹。

With air pollution becoming a global concern, scientists are committed to the study of air pollution. In the field of air pollution prediction, there have been good results in experimental research so far, but few studies have taken the weather forecast information and the properties of air pollution drifting. In our study, we proposed a wind-sensitive attention neural network model to learn the influence of wind direction and wind speed on the changes of spatial-temporal PM2.5 concentrations in neighboring areas. Then, preliminary predictions for PM2.5 are made by a long short-term memory neural network with neighboring pollutions which are ‘paid attention to’, and finally we apply ensemble learning method based on XGBoost to combine the preliminary predictions with weather forecast to make second phase prediction of PM2.5. The experiment is done using PM2.5 data and weather forecast data from EPA and CWB. Our results illustrate that the proposed method is superior to other methods, including linear regression, multi-layer perceptron, support vector regression, long short-term memory neural network and extreme gradient boosting algorithm, in predicting mid-term and long-term PM2.5 concentrations.

摘要 i
ABSTRACT ii
List of Tables v
List of Figures vi
1 Introduction 1
2 Related Work 5
2.1 Air Pollution 5
2.1.1 Air Pollution Impact 5
2.1.2 Air Pollution Factor 6
2.1.3 Air Pollution Observation 7
2.1.4 Air Pollution Forecasting 7
2.2 Time Series 9
2.2.1 Multi-layer Perceptron 9
2.2.2 Long Short-Term Memory Neural Network 9
2.3 Ensemble Learning 10
2.4 Attention 10
3 Proposed Approach 12
3.1 Overview 12
3.2 Preprocessing Method 13
3.2.1 One-hot Encoding for Wind Features 13
3.2.2 Making Time Series Features 14
3.3 Wind-Sensitive Attention Neural Network Model 15
3.3.1 Wind-Sensitive Attention Mechanism 16
3.3.2 Predictive Model of Long Short-Term Memory Neural Network 18
3.4 Ensemble Model for Second Phase Prediction 19
3.4.1 Extract Weather Forecast Features 19
3.4.2 XGBoost for Second Phase Prediction 19
4 Experiment and Evaluation 21
4.1 Dataset and Experiment Setup 21
4.2 Evaluation Metric 22
4.3 Comparison of Models 22
4.4 Parameters Setting for WANN model 23
4.4.1 Evaluation of the Number of Past Time Periods for PM2.5 24
4.4.2 Evaluation of the Number of Past Time Periods for Wind 25
4.4.3 Evaluation of the Number of Future Time Periods for Wind 26
4.5 Evaluation of Ensemble Methods 27
4.6 Evaluation of all Comparison Models 28
4.7 Discussion 29
5 Conclusions and Future Work 30
Reference 32

[1] M. Assaad, R. Boné, H. Cardot, A new boosting algorithm for improved time-series forecasting with recurrent neural networks, Information Fusion, 9 (1), 2008, pp. 41-55.
[2] D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473, 2014, pp.
[3] Y. Bai, Y. Li, X. Wang, J. Xie, C. Li, Air pollutants concentrations forecasting using back propagation neural network based on wavelet decomposition with meteorological conditions, Atmospheric pollution research, 7 (3), 2016, pp. 557-566.
[4] F. Biancofiore, M. Busilacchio, M. Verdecchia, B. Tomassetti, E. Aruffo, S. Bianco, S. Di Tommaso, C. Colangeli, G. Rosatelli, P. Di Carlo, Recursive neural network model for analysis and forecast of PM10 and PM2. 5, Atmospheric Pollution Research, 8 (4), 2017, pp. 652-659.
[5] J. Chen, J. Lu, J.C. Avise, J.A. DaMassa, M.J. Kleeman, A.P. Kaduwela, Seasonal modeling of PM2. 5 in California's San Joaquin Valley, Atmospheric environment, 92, 2014, pp. 182-190.
[6] T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785-794.
[7] L. Contreras, C. Ferri, Wind-sensitive interpolation of urban air pollution forecasts, Procedia Computer Science, 80, 2016, pp. 313-323.
[8] E. Erdem, J. Shi, ARMA based approaches for forecasting the tuple of wind speed and direction, Applied Energy, 88 (4), 2011, pp. 1405-1414.
[9] F.A. Gers, J. Schmidhuber, F. Cummins, Learning to forget: Continual prediction with LSTM, 1999, pp.
[10] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural computation, 9 (8), 1997, pp. 1735-1780.
[11] G. Hoek, R.M. Krishnan, R. Beelen, A. Peters, B. Ostro, B. Brunekreef, J.D. Kaufman, Long-term air pollution exposure and cardio-respiratory mortality: a review, Environmental Health, 12 (1), 2013, pp. 43.
[12] H.-P. Hsieh, S.-D. Lin, Y. Zheng, Inferring air quality for station location recommendation based on urban big data, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, pp. 437-446.
[13] P. Huang, J. Zhang, Y. Tang, L. Liu, Spatial and temporal distribution of PM2. 5 pollution in Xi’an City, China, International journal of environmental research and public health, 12 (6), 2015, pp. 6608-6625.
[14] P. Jiang, Q. Dong, P. Li, A novel hybrid strategy for PM2. 5 concentration analysis and prediction, Journal of environmental management, 196, 2017, pp. 443-457.
[15] J. Kleine Deters, R. Zalakeviciute, M. Gonzalez, Y. Rybarczyk, Modeling PM2. 5 urban pollution using machine learning and selected meteorological parameters, Journal of Electrical and Computer Engineering, 2017, 2017, pp.
[16] T.N. Krishnamurti, C. Kishtawal, Z. Zhang, T. LaRow, D. Bachiochi, E. Williford, S. Gadgil, S. Surendran, Multimodel ensemble forecasts for weather and seasonal climate, Journal of Climate, 13 (23), 2000, pp. 4196-4216.
[17] A. Kurt, A.B. Oktay, Forecasting air pollutant indicator levels with geographic models 3 days in advance using neural networks, Expert Systems with Applications, 37 (12), 2010, pp. 7986-7992.
[18] X. Li, Y. Cheng, G. Cong, L. Chen, Discovering Pollution Sources and Propagation Patterns in Urban Area, Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 1863-1872.
[19] X. Li, L. Peng, Y. Hu, J. Shao, T. Chi, Deep learning architecture for air quality predictions, Environmental Science and Pollution Research, 23 (22), 2016, pp. 22408-22417.
[20] X. Li, L. Peng, X. Yao, S. Cui, Y. Hu, C. You, T. Chi, Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation, Environmental Pollution, 231, 2017, pp. 997-1004.
[21] W. Lu, W. Wang, A.Y. Leung, S.-M. Lo, R.K. Yuen, Z. Xu, H. Fan, Air pollutant parameter forecasting using support vector machines, Neural Networks, 2002. IJCNN'02. Proceedings of the 2002 International Joint Conference on, 2002, pp. 630-635.
[22] X. Mao, T. Shen, X. Feng, Prediction of hourly ground-level PM2. 5 concentrations 3 days in advance using neural networks with satellite data in eastern China, Atmospheric Pollution Research, 8 (6), 2017, pp. 1005-1015.
[23] P.G. Nieto, E.F. Combarro, J. del Coz Díaz, E. Montañés, A SVM-based regression model to study the air quality at local scale in Oviedo urban area (Northern Spain): A case study, Applied Mathematics and Computation, 219 (17), 2013, pp. 8923-8937.
[24] B.T. Ong, K. Sugiura, K. Zettsu, Dynamically pre-trained deep recurrent neural networks using environmental monitoring data for predicting PM2. 5, Neural Computing and Applications, 27 (6), 2016, pp. 1553-1566.
[25] C.A. Pope III, R.T. Burnett, M.J. Thun, E.E. Calle, D. Krewski, K. Ito, G.D. Thurston, Lung cancer, cardiopulmonary mortality, and long-term exposure to fine particulate air pollution, Jama, 287 (9), 2002, pp. 1132-1141.
[26] Z. Qi, T. Wang, G. Song, W. Hu, X. Li, Z.M. Zhang, Deep Air Learning: Interpolation, Prediction, and Feature Analysis of Fine-grained Air Quality, IEEE Transactions on Knowledge and Data Engineering, 2018, pp.
[27] V. Reddy, P. Yedavalli, S. Mohanty, U. Nakhat, Deep Air: Forecasting Air Pollution in Beijing, China, pp.
[28] H. Sak, G. Yang, B. Li, W. Li, Modeling Dependence Dynamics of Air Pollution: Pollution Risk Simulation and Prediction of PM $ _ {2.5} $ Levels, arXiv preprint arXiv:1602.05349, 2016, pp.
[29] C. Song, J. He, L. Wu, T. Jin, X. Chen, R. Li, P. Ren, L. Zhang, H. Mao, Health burden attributable to ambient PM2. 5 in China, Environmental pollution, 223, 2017, pp. 575-586.
[30] J. Wang, S. Ogawa, Effects of meteorological conditions on PM2. 5 concentrations in Nagasaki, Japan, International journal of environmental research and public health, 12 (8), 2015, pp. 9089-9101.
[31] Y.-F. Xing, Y.-H. Xu, M.-H. Shi, Y.-X. Lian, The impact of PM2. 5 on the human respiratory system, Journal of thoracic disease, 8 (1), 2016, pp. E69.
[32] K. Yang, M.-c. Li, Z.-j. Chen, Q. Liao, Urban air pollution study based on GIS, Urban Remote Sensing Event, 2009 Joint, 2009, pp. 1-5.
[33] Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, E. Hovy, Hierarchical attention networks for document classification, Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016, pp. 1480-1489.
[34] C. Zhang, J. Yan, C. Li, X. Rui, L. Liu, R. Bie, On estimating air pollution from photos using convolutional neural network, Proceedings of the 2016 ACM on Multimedia Conference, 2016, pp. 297-301.
[35] Y. Zheng, F. Liu, H.-P. Hsieh, U-air: When urban air quality inference meets big data, Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 2013, pp. 1436-1444.
[36] Y. Zheng, X. Yi, M. Li, R. Li, Z. Shan, E. Chang, T. Li, Forecasting fine-grained air quality based on big data, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, pp. 2267-2276.