FUTURE-PROOFING MANUFACTURING: A DEEP DIVE INTO INNOVATIVE PROCESS MONITORING WITH THE KOOPMAN NEURAL NETWORK APPROACH
Abstract
Stochastic production systems (SPS) have demonstrated significant potential in various fields, including fermentation, pharmaceuticals, and composite material production. However, ensuring product quality in such scenarios is a critical challenge due to the stringent quality constraints. SPS is known for its intrinsic stochasticity and measurement uncertainty, making process monitoring essential but challenging.
The stochastic nature of SPS is exacerbated by external factors such as inputs, environmental conditions, and equipment state, all of which influence the final product's quality and performance. Moreover, the lack of precise in-situ measurements introduces additional noise into the available data. Consequently, effective process monitoring for SPS is indispensable.
Over the past few decades, several methods have been developed for SPS process monitoring. Multiway Principal Component Analysis (PCA) has been widely used due to its simplicity, low-dimensional computing space, and fast processing of high-dimensional data. However, it is limited by its linearity and cannot handle nonlinear dynamics. To address this limitation, the kernel method has been employed to map data into a high-dimensional feature space, making data linearly separable. Other methods, including an improved Independent Component Analysis (ICA), a kernel ICAPCA method, and a multiway kernel entropy ICA method, have been developed to capture nonlinear and non-Gaussian features in SPS data. Support Vector Machines (SVM) integrated with PCA or fuzzy reasoning have been used for anomaly detection in SPS. However, these methods struggle to handle the common characteristics of SPS data, such as nonlinearity, heavy-tailed distributions, and multimodality. Additionally, the tuning of hyperparameters in these methods can be cumbersome.
Keywords:
Stochastic Production Systems (SPS),, Process Monitoring, Principal Component Analysis (PCA),, Independent Component Analysis (ICA),, Support Vector Machines (SVM)Downloads
Published
Issue
Section
How to Cite
License
Copyright (c) 2023 Wang Xin Hua
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Wang, G., Haringa, C., Noorman, H., Chu, J., and Zhuang, Y. (2020). Developing a Computational Framework to Advance Bioprocess Scale-Up. Trends in Biotechnology, 38(8), 846-856.
Lu, J., Cao, Z., Zhao, C., and Gao, F. (2019). 110th Anniversary: An Overview on Learning-Based Model Predictive Control for Batch Processes. Industrial & Engineering Chemistry Research, 58(37), 17164-17173.
Jiang, Q., Wang, Z., Yan, S., and Cao, Z. (2022). Data-Driven Soft Sensing for Batch Processes Using Neural Network-Based Deep Quality-Relevant Representation Learning. IEEE Transactions on Artificial Intelligence.
Cao, Z., Yu, J., Wang, W., Lu, H., Xia, X., Xu, H., and Zhang, L. (2020). Multi-Scale Data-Driven Engineering for Biosynthetic Titer Improvement. Current Opinion in Biotechnology, 65, 205-212. [5] Gao, J., Feng, E., and Zhang, W. (2022). Modeling and Parameter Identification of Microbial Batch Fermentation under Environmental Disturbances. Applied Mathematical Modelling, 108, 205-219.
Soukoulis, C., Panagiotidis, P., Koureli, R., and Tzia, C. (2007). Industrial Yogurt Manufacture: Monitoring of Fermentation Process and Improvement of Final Product Quality. Journal of dairy science, 90(6), 2641-2654.
Sriramula, S., and Chryssanthopoulos, M. K. (2009). Quantification of Uncertainty Modelling in Stochastic Analysis of FRP Composites. Composites Part A: Applied Science and Manufacturing, 40(11), 1673-1684.
Lu, H., Plataniotis, K. N., and Venetsanopoulos, A. N. (2008). MPCA: Multilinear Principal Component Analysis of Tensor Objects. IEEE transactions on Neural Networks, 19(1), 18-39. [9] Lee, J. M., Yoo, C., and Lee, I. B. (2004). Fault Detection of Batch Processes Using Multiway Kernel Principal Component Analysis. Computers & Chemical Engineering, 28(9), 1837-1847.
Jia, Z. Y., Wang, P., and Gao, X. J. (2012). Process Monitoring and Fault Diagnosis of Penicillin Fermentation Based on Improved MICA. Advanced Materials Research, 591, 1783-1788.
Zhao, C., Gao, F., and Wang, F. (2009). Nonlinear Batch Process Monitoring Using Phase-Based Kernel-Independent Component Analysis−Principal Component Analysis (KICA−PCA). Industrial & Engineering Chemistry Research, 48(20), 9163-9174.
Peng, C., Chunhao, D., and Qiankun, Z. (2020). Fault Diagnosis of Microbial Pharmaceutical Fermentation Process with Non-Gaussian and Nonlinear Coexistence. Chemometrics and Intelligent Laboratory Systems, 199, 103931.
Yang, C., and Hou, J. (2016). Fed-Batch Fermentation Penicillin Process Fault Diagnosis and
Detection Based on Support Vector Machine. Neurocomputing, 190, 117-123.
Ding, J., Cao, Y., Mpofu, E., and Shi, Z. (2012). A Hybrid Support Vector Machine and Fuzzy Reasoning Based Fault Diagnosis and Rescue System for Stable Glutamate Fermentation. Chemical Engineering Research and Design, 90(9), 1197-1207.
Hornik, K., Stinchcombe, M., and White, H. (1989). Multilayer Feedforward Networks Are Universal Approximators. Neural Networks, 2(5), 359-366.
Shimizu, H., Yasuoka, K., Uchiyama, K., and Shioya, S. (1997). On-Line Fault Diagnosis for Optimal Rice a-Amylase Production Process of a Temperature-Sensitive Mutant of Saccharomyces Cerevisiae by an Autoassociative Neural Network. Journal of Fermentation and Bioengineering, 83(5), 435-442.
Lopes, J. A., and Menezes, J. C. (2004). Multivariate Monitoring of Fermentation Processes with Non-Linear Modelling Methods. Analytica Chimica Acta, 515(1), 101-108.
Yu, J., Zhang, C., and Wang, S. (2021). Multichannel One-Dimensional Convolutional Neural Network-Based Feature Learning for Fault Diagnosis of Industrial Processes. Neural Computing and Applications, 33, 3085-3104.
Chen, S., Yu, J., and Wang, S. (2020). One-Dimensional Convolutional Auto-Encoder-Based Feature Learning for Fault Diagnosis of Multivariate Processes. Journal of Process Control, 87, 54-67. [20] Peng, C., Lu, R., Kang, O., and Kai, W. (2020). Batch Process Fault Detection for Multi-Stage Broad Learning System. Neural Networks, 129, 298-312.
Chen, H., Liu, Z., Alippi, C., Huang, B., and Liu, D. (2022). Explainable Intelligent Fault Diagnosis for Nonlinear Dynamic Systems: From Unsupervised to Supervised Learning. IEEE Transactions on Neural Networks and Learning Systems.
Chen, H., Chai, Z., Dogru, O., Jiang, B., and Huang, B. (2021). Data-Driven Designs of Fault Detection Systems Via Neural Network-Aided Learning. IEEE Transactions on Neural Networks and Learning Systems, 33(10), 5694-5705.
Sherstinsky, A. (2020). Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. Physica D: Nonlinear Phenomena, 404, 132306.
Zhang, M., Li, X., and Wang, R. (2021). Incipient Fault Diagnosis of Batch Process Based on Deep Time Series Feature Extraction. Arabian Journal for Science and Engineering, 1-12.
Ren, J., and Ni, D. (2020). A Batch-Wise LSTM-Encoder Decoder Network for Batch Process Monitoring. Chemical Engineering Research and Design, 164, 102-112.
Koopman, B. O. (1931). Hamiltonian Systems and Transformation in Hilbert Space. Proceedings of the National Academy of Sciences, 17(5), 315-318.
Brunton, S. L. (2019). Notes on Koopman Operator Theory. Universität Von Washington, Department of Mechanical Engineering, Zugriff, 30.
Rowley, C. W., Mezić, I., Bagheri, S., Schlatter, P., and Henningson, D. S. (2009). Spectral Analysis of Nonlinear Flows. Journal of Fluid Mechanics, 641, 115-127.
Schmid, P. J. (2010). Dynamic Mode Decomposition of Numerical and Experimental Data. Journal of Fluid Mechanics, 656, 5-28.
Williams, M. O., Kevrekidis, I. G., and Rowley, C. W. (2015). A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition. Journal of Nonlinear Science, 25, 13071346.
Korda, M., and Mezić, I. (2018). Linear Predictors for Nonlinear Dynamical Systems: Koopman Operator Meets Model Predictive Control. Automatica, 93, 149-160.
Brunton, S. L., Brunton, B. W., Proctor, J. L., and Kutz, J. N. (2016). Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control. Plos One, 11(2), e0150171.
Lusch, B., Kutz, J. N., and Brunton, S. L. (2018). Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics. Nature Communications, 9(1), 4950.
Yeung, E., Kundu, S., and Hodas, N. (2019, July). Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems. In 2019 American Control Conference (ACC) (pp. 4832-4839). IEEE.
Dubey, R., Samantaray, S. R., Panigrahi, B. K., and Venkoparao, V. G. (2016). Koopman Analysis Based Wide-Area Back-Up Protection and Faulted Line Identification for Series-Compensated Power Network. IEEE Systems Journal, 12(3), 2634-2644.
Dang, Z., Lv, Y., Li, Y., and Wei, G. (2018). Improved Dynamic Mode Decomposition and Its Application to Fault Diagnosis of Rolling Bearing. Sensors, 18(6), 1972.
Cheng, C., Ding, J., and Zhang, Y. (2020). A Koopman Operator Approach for Machinery Health Monitoring and Prediction with Noisy and Low-Dimensional Industrial Time Series. Neurocomputing, 406, 204-214.
Liu, B., Xiao, Y., Cao, L., Hao, Z., and Deng, F. (2013). Svdd-Based Outlier Detection on Uncertain Data. Knowledge and Information Systems, 34, 597-618.
Larsson, A. J., Johnsson, P., Hagemann-Jensen, M., Hartmanis, L., Faridani, O. R., Reinius, B., and Sandberg, R. (2019). Genomic Encoding of Transcriptional Burst Kinetics. Nature, 565(7738), 251254.
Cao, Z., and Grima, R. (2020). Analytical Distributions for Detailed Models of Stochastic Gene Expression in Eukaryotic Cells. Proceedings of the National Academy of Sciences, 117(9), 4682-4692. [41] Gillespie, D. T. (1977). Exact Stochastic Simulation of Coupled Chemical Reactions. The Journal of Physical Chemistry, 81(25), 2340-2361.
Fu, X., Zhou, X., Gu, D., Cao, Z., and Grima, R. (2022). DelaySSAToolkit. jl: Stochastic Simulation of Reaction Systems with Time Delays in Julia. Bioinformatics, 38(17), 4243-4245.
Brunton, S. L., Budišić, M., Kaiser, E., and Kutz, J. N. (2021). Modern Koopman Theory for Dynamical Systems. arXiv preprint arXiv:2102.12086.
Cao, Z., and Grima, R. (2018). Linear Mapping Approximation of Gene Regulatory Networks with Stochastic Dynamics. Nature Communications, 9(1), 3305.
