臺灣博碩士論文加值系統

Nowadays, in this era the direct sales from manufacturer to the customer is a common thing, so it will not need to get through by retailer. In the direct sales channel, manufacturers sell products directly to customers through the internet, their owned stores or mail order to reduce the distribution cost. As the increasing the demand, the lead time is longer too, make some customers cancel the demand. It makes the manufacturers facing stochastic environment that they need to decide the right amount to produce. In this research we purpose using reinforcement learning approach to decide the production per period. There are several factors to see the performance of reinforcement learning that the price, standard deviation, lead time dependent demand factor, production cost, backorder cost, and inventory cost. The result shows that using the reinforcement learning succeed to reduce the backorder by 17% and gain more profit by 2.62%. it concludes that whatever the design of different instances settings, there are still improvement on the profit and the inventory level.

Nowadays, in this era the direct sales from manufacturer to the customer is a common thing, so it will not need to get through by retailer. In the direct sales channel, manufacturers sell products directly to customers through the internet, their owned stores or mail order to reduce the distribution cost. As the increasing the demand, the lead time is longer too, make some customers cancel the demand. It makes the manufacturers facing stochastic environment that they need to decide the right amount to produce. In this research we purpose using reinforcement learning approach to decide the production per period. There are several factors to see the performance of reinforcement learning that the price, standard deviation, lead time dependent demand factor, production cost, backorder cost, and inventory cost. The result shows that using the reinforcement learning succeed to reduce the backorder by 17% and gain more profit by 2.62%. it concludes that whatever the design of different instances settings, there are still improvement on the profit and the inventory level.

TABLE OF CONTENTS

CHAPTER 1 INTRODUCTION 1
1.1 Research Background 1
1.2 Research Motivation 2
1.3 Research Objective 3
1.4 Research Structure 4
CHAPTER 2 LITERATURE REVIEW 6
2.1 Lead Time Dependent Demand 6
2.2 Machine Learning 7
2.3 Reinforcement Learning 9
2.4 Application of Reinforcement Learning 10
2.5 Summary 11
CHAPTER 3 RESEARCH METHODOLOGY 12
3.1 Problem Description 12
3.2 Notation 12
3.3 Basic Model and Assumption 13
3.4 Reinforcement Learning Model 15
CHAPTER 4 RESULT AND DISCUSSION 22
4.1 Building the Q table 22
4.2 Parameter Setting 25
4.3 Evaluate the performance of Reinforcement Learning 30
4.4 Sensitivity Analysis 42
CHAPTER 5 CONCLUSION AND FUTURE RESEARCH 52

REFERENCES

Altendorfer, K., 2017. Relation between lead time dependent demand and capacity flexibility in a two-stage supply chain with lost sales. International Journal of Production Economics, 194, 13-24.
Armstrong, G., Adam, S., Denize, S. & Kotler, P., 2014. Principles of marketing: Pearson Australia.
Balasubramanian, S., 1998. Mail versus mall: A strategic analysis of competition between direct marketers and conventional retailers. Marketing Science, 17 (3), 181-195.
Ben-Daya, M.A. & Raouf, A., 1994. Inventory models involving lead time as a decision variable. Journal of the Operational Research Society, 45 (5), 579-582.
Chi, H.-M., Ersoy, O.K., Moskowitz, H. & Ward, J., 2007. Modeling and optimizing a vendor managed replenishment system using machine learning and genetic algorithms. European Journal of Operational Research, 180 (1), 174-193.
Gosavi, A., Ozkaya, E. & Kahraman, A.F., 2007. Simulation optimization for revenue management of airlines with cancellations and overbooking. Or Spectrum, 29 (1), 21-38.
Govindu, R. & Chinnam, R.B., 2007. Mascf: A generic process-centered methodological framework for analysis and design of multi-agent supply chain systems. Computers & Industrial Engineering, 53 (4), 584-609.
Gunasekaran, A., Lai, K.-H. & Cheng, T.E., 2008. Responsive supply chain: A competitive strategy in a networked economy. Omega, 36 (4), 549-564.
Harmon, M.E. & Harmon, S.S., 1996. Reinforcement learning: A tutorial. WL/AAFC, WPAFB Ohio, 45433.
Hua, G., Wang, S. & Cheng, T.E., 2010. Price and lead time decisions in dual-channel supply chains. European journal of operational research, 205 (1), 113-126.
Jiang, C. & Sheng, Z., 2009. Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain system. Expert Systems with Applications, 36 (3), 6520-6526.
Kaelbling, L.P., Littman, M.L. & Moore, A.W., 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4, 237-285.
Kara, A. & Dogan, I., 2018. Reinforcement learning approaches for specifying ordering policies of perishable inventory systems. Expert Systems with Applications, 91, 150-158.
Liang, W.-Y. & Huang, C.-C., 2006. Agent-based demand forecast in multi-echelon supply chain. Decision support systems, 42 (1), 390-407.
Liao, C.-J. & Shyu, C.H.-H., 1991. Stochastic inventory model with controllable lead time. International Journal of Systems Science, 22 (11), 2347-2354.
Ouyang, L.-Y., Yeh, N.-C. & Wu, K.-S., 1996. Mixture inventory model with backorders and lost sales for variable lead time. Journal of the Operational Research Society, 47 (6), 829-832.
Rana, R. & Oliveira, F.S., 2015. Dynamic pricing policies for interdependent perishable products or services using reinforcement learning. Expert Systems with Applications, 42 (1), 426-436.
Ray, S. & Jewkes, E.M., 2004. Customer lead time management when both demand and price are lead time sensitive. European Journal of operational research, 153 (3), 769-781.
Russell, S.J., Norvig, P., Canny, J.F., Malik, J.M. & Edwards, D.D., 2003. Artificial intelligence: A modern approach: Prentice hall Upper Saddle River.
Shin, M., Ryu, K. & Jung, M., 2012. Reinforcement learning approach to goal-regulation in a self-evolutionary manufacturing system. Expert Systems with Applications, 39 (10), 8736-8743.
Simchi-Levi, D. & Kaminsky, P., 2008. Simchi-levi. Designing and managing the supply chain: concepts, strategies and case studies.
Simon, P., 2013. Too big to ignore: The business case for big data: John Wiley & Sons.
Sui, Z., Gosavi, A. & Lin, L., 2010. A reinforcement learning approach for inventory replenishment in vendor-managed inventory systems with consignment inventory. Engineering Management Journal, 22 (4), 44-53.
Sutton, R.S. & Barto, A.G., 1998. Reinforcement learning: An introduction: MIT press Cambridge.
Tsai, C.-Y., Chu, W.-F. & Tu, C.-Y., Year. Analysis of sales strategy with lead-time sensitive demanded.^eds. Proceedings of the Institute of Industrial Engineers Asian Conference 2013Springer, 985-992.
Yang, B. & Geunes, J., 2007. Inventory and lead time planning with lead-time-sensitive demand. IIE transactions, 39 (5), 439-452.
Zhang, W. & Dietterich, T.G., Year. A reinforcement learning approach to job-shop schedulinged.^eds. IJCAICiteseer, 1114-1120.