PriMera Scientific Engineering (ISSN: 2834-2550)

Research Article

Volume 9 Issue 1

Integrated Hierarchical Reinforcement Learning for Enhanced CMP Manufacturing Efficiency

Andy Yendi Tsen*

June 30, 2026

DOI : 10.56831/PSEN-09-279

View PDF

Abstract

This study demonstrates that applying artificial intelligence (AI) to chemical-mechanical planarization (CMP) process control significantly reduces rework rates and enhances manufacturing efficiency. We propose a hierarchical AI CMP controller featuring an auto-tuning capability. This integrated hierarchical reinforcement learning (iHRL) framework is designed to mitigate CMP variations arising from removal rate decay and pattern density fluctuations. The iHRL agent utilizes a structured wafer-to-wafer and lot-to-lot control architecture. Results demonstrate that the self-learning auto-tuning mechanism effectively minimizes the CMP rework rate. Furthermore, comprehensive simulations indicate that the proposed controller enhances average manufacturing efficiency by more than double digits compared to the conventional run-to-run controller.

Keywords: semiconductor manufacturing; chemical mechanical planarization; artificial intelligence

References

  1. A-C Lee, Y-R Pan and M-T Hsieh. “Output disturbance observer structure applied to run-to-run control for semiconductor manufacturing”. IEEE Transactions on Semiconductor Manufacturing 24.1 (2011): 27-43.
  2. WJ Campbell., et al. “A comparison of run-to-run control algorithms”. in Proceedings of the 2002 American Control Conference (IEEE Cat. No. CH37301) IEEE 3 (2002): 2150-2155.
  3. AC Lee, TW Kuo and ZL Lee. “Modified double EWMA approach for mixed product run-to-run CMP process control”. Advanced Materials Research 314-316 (2011): 2504-2511.
  4. E Sachs, A Hu and A Ingolfsson. “Run by run process control: Combining SPC and feedback control”. IEEE Transactions on Semiconductor Manufacturing 8.1 (1995): 26-43.
  5. Y Zheng., et al. “Model quality evaluation in semiconductor manufacturing process with EWMA run-to-run control”. IEEE Transactions on Semiconductor Manufacturing 30.1 (2017): 8-16.
  6. TP Lillicrap., et al. “Continuous control with deep reinforcement learning”. arXiv preprint arXiv:1509.02971, (2015).
  7. MM Thomas, B Joseph and JL Kardos. “Batch chemical process quality control applied to curing of composite materials”. AIChE journal 43.10 (1997): 2535-2545.
  8. AYD Tsen., et al. “Predictive control of quality in batch polymerization using hybrid ANN models”. AIChE Journal 42.2 (1996): 455-465.
  9. TH Smith and DS Boning. “A self-tuning EWMA controller utilizing artificial neural network function approximation techniques”. in IEEE Transactions on Components, Packaging, and Manufacturing Technology: Part C 20.2 (1997): 121-132.
  10. Y Li, J Du and W Jiang. “Reinforcement learning for process control with application in semiconductor manufacturing”. IISE Transactions 56.6 (2024): 585-599.
  11. A Yen-Di Tsen and T-L Chen. “Reinforcement learning chemical-mechanical polishing run-to-run controller”. in 2023 IEEE 5th Eurasia Conference on IOT, Communication and Engineering (ECICE). IEEE (2023): 732-735.
  12. AY Tsen, T-L Chen and K-S Hsu. “Deep Hierarchical Reinforcement Learning CMP Run to Run Control”. in International Conference on Flexible Automation and Intelligent Manufacturing. Springer (2024): 58-65.
  13. AG Barto and S Mahadevan. “Recent advances in hierarchical reinforcement learning”. Discrete event dynamic systems 13.4 (2003): 341-379.
  14. S Pateria., et al. “Hierarchical reinforcement learning: A comprehensive survey”. ACM Computing Surveys (CSUR) 54.5 (2022): 1-35.
  15. JM Steigerwald, SP Murarka and RJ Gutmann. Chemical mechanical planarization of microelectronic materials. John Wiley & Sons (1997).
  16. H Lee, H Jeong and D Dornfeld. “Semi-empirical material removal rate distribution model for SiO2 chemical mechanical polishing (CMP) processes”. Precision Engineering 37.2 (2013): 483-490.
  17. BE Stine., et al. “Rapid characterization and modeling of pattern-dependent variation in chemical-mechanical polishing”. IEEE Transactions on Semiconductor manufacturing 11.1 (1998): 129-140.
  18. D Ouma., et al. “An integrated characterization and modeling methodology for CMP dielectric planarization”. in Proceedings of the IEEE 1998 International Interconnect Technology Conference (Cat. No. 98EX102). IEEE (1998): 67-69.
  19. A Maury., et al. “A modification to Preston’s equation and impact on pattern density effect modeling”. Advance metallization and interconnect system for ULSI application (1997).