A Reinforcement Learning Approach for Improved Photolithography Schedules

A Reinforcement Learning (RL) model is applied for photolithography schedules with direct consideration of reentrant visits. The photolithography process is mainly regarded as a bottleneck process in semiconductor manufacturing, and improving its schedules would result in better performances. Most RL-based research do not consider revisits directly or guarantee convergence. A simplified discrete event simulation model of a fabrication facility is built, and a tabular Q-learning agent is embedded into the model to learn through scheduling. The learning environment considers states and actions consisting of information on reentrant flows. The agent dynamically chooses one rule from a pre-defined rule set to dispatch lots. The set includes the earliest stage first, the latest stage first, and 8 more composite rules. Finally, the proposed RL approach is compared with 7 single and 8 hybrid rules. The method presents a validated approach in terms of overall average cycle times.