Selective maintenance optimization using machine learning and agent based approach

Abstract

A Selective Maintenance (SM) policy is used for the maintenance of equipment that works in mission mode. Most of the approaches for Selective Maintenance Optimization (SMO) require higher computation time to determine the maintenance policy, which is undesirable. The advent of machine learning opens up new horizons for developing novel approaches that have the potential to substantially reduce computation time in SMO. The rapid rise in the use of sensors and computing infrastructure is transforming conventional industrial systems into smart machines. There is an opportunity to embrace this smartness in every aspect of industrial systems. Maintenance planning is one such inherent aspect. Technologies like multi-agent systems are making a move from centralized decision making to distributed realization of decision making. This project proposes to develop a novel Reinforcement Learning (RL) based methodology and a distributed algorithm for intelligent maintenance planning to optimize the selective maintenance problem. As part of the RL based methodology, a temporal difference learning algorithm - Q-Learning is used to solve this optimization problem where the agent chooses a policy at the end of an epoch, based on the updated Q-Values. Different heuristics are embedded with the methodology to effectively determine the optimal policy, of which one smartly reduces the solution space and the other aids in increasing the agent’s intelligence based on the reward policy. The objective of the maintenance optimization problem is to maximize the system reliability and it is formulated as a Semi-Markov decision process (SMDP). The reward function is also defined in a way that the agent will try to determine the best strategy with minimum consumption of maintenance resources. The algorithm designed for agent based distributed maintenance planning fits into the Industrial Internet of Things (IIoT) paradigm and revolves around the idea of having individual agents to make maintenance decisions for respective subsystems and an overall coordinating agent that will decide the optimal policy from the preferences given by the subsystem level agent and decides what maintenance policy best for the enterprise The efficiency of the developed algorithms is demonstrated by applying it to a benchmark multi-state industrial system for coal transportation. The results accentuate the supremacy of the developed RL based algorithm and agent based distributed approach over the commonly used methods like the enumeration approach and genetic algorithm based approach.