Publications

Scientific publications of the institute in the last two years

2025

• Sign Diversity: A Method for Measuring Diversity in Base Learner Selection for Ensemble Regression. P, Farzad Rezazadeh; Olfatbakhsh, Emad; Kroll, Andreas. In 2025 IEEE Symposium Series on Computational Intelligence (SSCI). 2025.
  • [ BibTeX ]
  @article{rezazadeh2025SSCI, author = {P, Farzad Rezazadeh and Olfatbakhsh, Emad and Kroll, Andreas}, journal = {2025 IEEE Symposium Series on Computational Intelligence (SSCI)}, keywords = {isac-www}, note = {accepted}, title = {Sign Diversity: A Method for Measuring Diversity in Base Learner Selection for Ensemble Regression}, year = 2025 }

2024

• Data-Driven Model Predictive Control of Battery Storage Units. Lipka, Johannes B.; Hans, Christian A. In 2024 American Control Conference (ACC), bll 2230–2235. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  In many state-of-the-art control approaches for power systems with storage units, an explicit model of the storage dynamics is required. With growing numbers of storage units, identifying these dynamics can be cumbersome. This paper employs recent data-driven control approaches that do not require an explicit identification step. Instead, they use measured input/output data in control formulations. In detail, we propose an economic data-driven model predictive control (MPC) scheme to operate a small power system with input-nonlinear battery dynamics. First, a linear data-driven MPC approach that uses a slack variable to account for plant-model-mismatch is proposed. In a second step, an input-nonlinear data-driven MPC scheme is deduced. Comparisons with a reference indicate that the linear data-driven MPC approximates the nonlinear plant in an acceptable manner. Even better results, however, can be obtained with the input-nonlinear data-driven MPC scheme which provides increased prediction accuracy.

  @inproceedings{10644376, abstract = {In many state-of-the-art control approaches for power systems with storage units, an explicit model of the storage dynamics is required. With growing numbers of storage units, identifying these dynamics can be cumbersome. This paper employs recent data-driven control approaches that do not require an explicit identification step. Instead, they use measured input/output data in control formulations. In detail, we propose an economic data-driven model predictive control (MPC) scheme to operate a small power system with input-nonlinear battery dynamics. First, a linear data-driven MPC approach that uses a slack variable to account for plant-model-mismatch is proposed. In a second step, an input-nonlinear data-driven MPC scheme is deduced. Comparisons with a reference indicate that the linear data-driven MPC approximates the nonlinear plant in an acceptable manner. Even better results, however, can be obtained with the input-nonlinear data-driven MPC scheme which provides increased prediction accuracy.}, author = {Lipka, Johannes B. and Hans, Christian A.}, booktitle = {2024 American Control Conference (ACC)}, keywords = {isac-www}, month = {July}, pages = {2230-2235}, title = {Data-Driven Model Predictive Control of Battery Storage Units}, year = 2024 }
• Casimir Effect in MEMS: Materials, Geometries, and Metrologies—A Review. Elsaka, Basma; Yang, Xiaohui; Kästner, Philipp; Dingel, Kristina; Sick, Bernhard; Lehmann, Peter; Buhmann, Stefan Yoshi; Hillmer, Hartmut. In Materials, 17(14), bl 3393. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Casimir force densities, i.e., force per area, become very large if two solid material surfaces come closer together to each other than 10 nm. In most cases, the forces are attractive. In some cases, they can be repulsive depending on the solid materials and the fluid medium in between. This review provides an overview of experimental and theoretical studies that have been performed and focuses on four main aspects: (i) the combinations of different materials, (ii) the considered geometries, (iii) the applied experimental measurement methodologies and (iv) a novel self-assembly methodology based on Casimir forces. Briefly reviewed is also the influence of additional parameters such as temperature, conductivity, and surface roughness. The Casimir effect opens many application possibilities in microelectromechanical systems (MEMS) and nanoelectromechanical systems (NEMS), where an overview is also provided. The knowledge generation in this fascinating field requires interdisciplinary approaches to generate synergetic effects between technological fabrication metrology, theoretical simulations, the establishment of adequate models, artificial intelligence, and machine learning. Finally, multiple applications are addressed as a research roadmap.

  @article{elsaka2024casimir, abstract = {Casimir force densities, i.e., force per area, become very large if two solid material surfaces come closer together to each other than 10 nm. In most cases, the forces are attractive. In some cases, they can be repulsive depending on the solid materials and the fluid medium in between. This review provides an overview of experimental and theoretical studies that have been performed and focuses on four main aspects: (i) the combinations of different materials, (ii) the considered geometries, (iii) the applied experimental measurement methodologies and (iv) a novel self-assembly methodology based on Casimir forces. Briefly reviewed is also the influence of additional parameters such as temperature, conductivity, and surface roughness. The Casimir effect opens many application possibilities in microelectromechanical systems (MEMS) and nanoelectromechanical systems (NEMS), where an overview is also provided. The knowledge generation in this fascinating field requires interdisciplinary approaches to generate synergetic effects between technological fabrication metrology, theoretical simulations, the establishment of adequate models, artificial intelligence, and machine learning. Finally, multiple applications are addressed as a research roadmap.}, author = {Elsaka, Basma and Yang, Xiaohui and Kästner, Philipp and Dingel, Kristina and Sick, Bernhard and Lehmann, Peter and Buhmann, Stefan Yoshi and Hillmer, Hartmut}, journal = {Materials}, keywords = {isac-www}, number = 14, pages = 3393, title = {Casimir Effect in MEMS: Materials, Geometries, and Metrologies—A Review}, volume = 17, year = 2024 }
• Maintaining Strong Structural Controllability for Multi-Agent Systems with Varying Number of Agents. V. Schmidtke, V.; Al-Maqdad, R.K.; Stursberg, O. In 63nd IEEE Conference on Decision and Control, 63nd I. C. on Decision; Control (reds.). 2024.
  • [ Abstract ]
  • [ BibTeX ]
  This paper investigates strong structural controllability for multi-agent systems (MAS) with dynamically joining and leaving agents. Changing sets of agents can compromise controllability if only a subset of agents is equipped with control inputs. Based on the graph-theoretical concept of zero forcing, conditions are derived under which MAS with joining and leaving agents can maintain strong structural controllability without needing to reconfigure the leader set. Two methods are proposed: The first, employs so-called zero forcing chains, to provide definitive statements on maintaining SSC. The second is based on forcing agents and offers a quick assessment of feasibility for joining or leaving, but confirms SSC only in specific cases.

  @article{vschmidtke2024maintaining, abstract = {This paper investigates strong structural controllability for multi-agent systems (MAS) with dynamically joining and leaving agents. Changing sets of agents can compromise controllability if only a subset of agents is equipped with control inputs. Based on the graph-theoretical concept of zero forcing, conditions are derived under which MAS with joining and leaving agents can maintain strong structural controllability without needing to reconfigure the leader set. Two methods are proposed: The first, employs so-called zero forcing chains, to provide definitive statements on maintaining SSC. The second is based on forcing agents and offers a quick assessment of feasibility for joining or leaving, but confirms SSC only in specific cases.}, author = {V. Schmidtke, V. and Al-Maqdad, R.K. and Stursberg, O.}, editor = {63nd IEEE Conference on Decision and Control}, journal = {63nd IEEE Conference on Decision and Control}, keywords = {isac-www}, title = {Maintaining Strong Structural Controllability for Multi-Agent Systems with Varying Number of Agents}, year = 2024 }
• Location based Probabilistic Load Forecasting of EV Charging Sites: Deep Transfer Learning with Multi-Quantile Temporal Convolutional Network. Ali, Mohammad Wazed; Mustafa, Mohammed Asif bin; Shuvo, Md. Aukerul Moin; Sick, Bernhard. In arXiv e-prints, bl arXiv:2409.11862. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Electrification of vehicles is a potential way of reducing fossil fuel usage and thus lessening environmental pollution. Electric Vehicles (EVs) of various types for different transport modes (including air, water, and land) are evolving. Moreover, different EV user groups (commuters, commercial or domestic users, drivers) may use different charging infrastructures (public, private, home, and workplace) at various times. Therefore, usage patterns and energy demand are very stochastic. Characterizing and forecasting the charging demand of these diverse EV usage profiles is essential in preventing power outages. Previously developed data-driven load models are limited to specific use cases and locations. None of these models are simultaneously adaptive enough to transfer knowledge of day-ahead forecasting among EV charging sites of diverse locations, trained with limited data, and cost-effective. This article presents a location-based load forecasting of EV charging sites using a deep Multi-Quantile Temporal Convolutional Network (MQ-TCN) to overcome the limitations of earlier models. We conducted our experiments on data from four charging sites, namely Caltech, JPL, Office-1, and NREL, which have diverse EV user types like students, full-time and part-time employees, random visitors, etc. With a Prediction Interval Coverage Probability (PICP) score of 93.62%, our proposed deep MQ-TCN model exhibited a remarkable 28.93% improvement over the XGBoost model for a day-ahead load forecasting at the JPL charging site. By transferring knowledge with the inductive Transfer Learning (TL) approach, the MQ-TCN model achieved a 96.88% PICP score for the load forecasting task at the NREL site using only two weeks of data.

  @article{ali2024location, abstract = {Electrification of vehicles is a potential way of reducing fossil fuel usage and thus lessening environmental pollution. Electric Vehicles (EVs) of various types for different transport modes (including air, water, and land) are evolving. Moreover, different EV user groups (commuters, commercial or domestic users, drivers) may use different charging infrastructures (public, private, home, and workplace) at various times. Therefore, usage patterns and energy demand are very stochastic. Characterizing and forecasting the charging demand of these diverse EV usage profiles is essential in preventing power outages. Previously developed data-driven load models are limited to specific use cases and locations. None of these models are simultaneously adaptive enough to transfer knowledge of day-ahead forecasting among EV charging sites of diverse locations, trained with limited data, and cost-effective. This article presents a location-based load forecasting of EV charging sites using a deep Multi-Quantile Temporal Convolutional Network (MQ-TCN) to overcome the limitations of earlier models. We conducted our experiments on data from four charging sites, namely Caltech, JPL, Office-1, and NREL, which have diverse EV user types like students, full-time and part-time employees, random visitors, etc. With a Prediction Interval Coverage Probability (PICP) score of 93.62%, our proposed deep MQ-TCN model exhibited a remarkable 28.93% improvement over the XGBoost model for a day-ahead load forecasting at the JPL charging site. By transferring knowledge with the inductive Transfer Learning (TL) approach, the MQ-TCN model achieved a 96.88% PICP score for the load forecasting task at the NREL site using only two weeks of data.}, author = {Ali, Mohammad Wazed and Mustafa, Mohammed Asif bin and Shuvo, Md. Aukerul Moin and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2409.11862}, title = {Location based Probabilistic Load Forecasting of EV Charging Sites: Deep Transfer Learning with Multi-Quantile Temporal Convolutional Network.}, year = 2024 }
• The Interplay of Uncertainty Modeling and Deep Active Learning: An Empirical Analysis in Image Classification. Huseljic, Denis; Herde, Marek; Nagel, Yannick; Rauch, Lukas; Strimaitis, Paulius; Sick, Bernhard. In Transactions on Machine Learning Research. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Deep active learning (AL) seeks to reduce the annotation costs required for training deep neural networks (DNNs). Often, deep AL strategies focus on instances where the predictive uncertainty of a DNN is high. Furthermore, Bayesian concepts to model uncertainty are frequently adopted. Despite considerable research, a detailed analysis of the role of uncertainty in deep AL is still missing, especially regarding aleatoric and epistemic uncertainty, both related to predictive uncertainty. This article provides an in-depth empirical study analyzing the interplay of uncertainty and deep AL in image classification. Our study investigates four hypotheses that provide an intuitive understanding of the effects of accurately estimating aleatoric and epistemic uncertainty on existing uncertainty-based AL strategies but also, in the opposite direction, the impact of uncertainty-based AL on the quality of uncertainty estimates that are needed in many applications. By analyzing these hypotheses on synthetic and real-world data, we find that accurate aleatoric estimates can even impair instance selection, while accurate epistemic estimates have negligible effects. Moreover, we provide a publicly available toolbox for deep AL with various models and strategies to facilitate further research and practical applications. Code is available at https://github.com/dhuseljic/dal-toolbox.

  @article{huseljic2024interplay, abstract = {Deep active learning (AL) seeks to reduce the annotation costs required for training deep neural networks (DNNs). Often, deep AL strategies focus on instances where the predictive uncertainty of a DNN is high. Furthermore, Bayesian concepts to model uncertainty are frequently adopted. Despite considerable research, a detailed analysis of the role of uncertainty in deep AL is still missing, especially regarding aleatoric and epistemic uncertainty, both related to predictive uncertainty. This article provides an in-depth empirical study analyzing the interplay of uncertainty and deep AL in image classification. Our study investigates four hypotheses that provide an intuitive understanding of the effects of accurately estimating aleatoric and epistemic uncertainty on existing uncertainty-based AL strategies but also, in the opposite direction, the impact of uncertainty-based AL on the quality of uncertainty estimates that are needed in many applications. By analyzing these hypotheses on synthetic and real-world data, we find that accurate aleatoric estimates can even impair instance selection, while accurate epistemic estimates have negligible effects. Moreover, we provide a publicly available toolbox for deep AL with various models and strategies to facilitate further research and practical applications. Code is available at https://github.com/dhuseljic/dal-toolbox.}, author = {Huseljic, Denis and Herde, Marek and Nagel, Yannick and Rauch, Lukas and Strimaitis, Paulius and Sick, Bernhard}, journal = {Transactions on Machine Learning Research}, keywords = {isac-www}, title = {The Interplay of Uncertainty Modeling and Deep Active Learning: An Empirical Analysis in Image Classification}, year = 2024 }
• Enhancing Maritime Behaviour Analysis through Novel Feature Engineering and Digital Shadow Modelling: A Case Study in Kiel Fjord. Al-Falouji, Ghassan; Gao, Shang; Haschke, Lukas; Nowotka, Dirk; Tomforde, Sven. In International Conference on Architecture of Computing Systems (ARCS), bll 97–111. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  With the continuous evolution of maritime technology, there is a growing need for analysing and modelling vessel behaviour in complex waterway systems. This paper presents an extension and utilisation of the Surface Vessel Nautical Behaviour Analysis (SV-NBA) framework for in-depth spatio-temporal analysis of maritime surface vessels’ behaviour in Kiel Fjord. Leveraging one year of collected Automatic Identification System (AIS) data, we extracted features from the recorded data. Three feature sets are generated and compared using expert-knowledge fea- tures, methods for feature selection, and Denoising Autoencoder latent space representation. Behaviour modelling and analysis utilises clustered data from the three feature sets, employing Gaussian Mixture Models (GMM). The trained GMM models serve as digital shadows, enabling storage-efficient representation of vessel behaviour and facilitates appli- cations such as online situational awareness and marine-traffic manage- ment. These digital shadows serve as observer-layer in a dynamic au- tonomous system of system, offering insights into maritime activities and enhancing navigation safety in busy waterways. This research contributes to the advancement of autonomous navigation systems and supports ef- ficient maritime traffic management strategies.

  @inproceedings{AlFalouji2024Behaviour, abstract = {With the continuous evolution of maritime technology, there is a growing need for analysing and modelling vessel behaviour in complex waterway systems. This paper presents an extension and utilisation of the Surface Vessel Nautical Behaviour Analysis (SV-NBA) framework for in-depth spatio-temporal analysis of maritime surface vessels’ behaviour in Kiel Fjord. Leveraging one year of collected Automatic Identification System (AIS) data, we extracted features from the recorded data. Three feature sets are generated and compared using expert-knowledge fea- tures, methods for feature selection, and Denoising Autoencoder latent space representation. Behaviour modelling and analysis utilises clustered data from the three feature sets, employing Gaussian Mixture Models (GMM). The trained GMM models serve as digital shadows, enabling storage-efficient representation of vessel behaviour and facilitates appli- cations such as online situational awareness and marine-traffic manage- ment. These digital shadows serve as observer-layer in a dynamic au- tonomous system of system, offering insights into maritime activities and enhancing navigation safety in busy waterways. This research contributes to the advancement of autonomous navigation systems and supports ef- ficient maritime traffic management strategies.}, author = {Al-Falouji, Ghassan and Gao, Shang and Haschke, Lukas and Nowotka, Dirk and Tomforde, Sven}, booktitle = {International Conference on Architecture of Computing Systems (ARCS)}, keywords = {Encoders}, pages = {97–111}, publisher = {Springer}, title = {Enhancing Maritime Behaviour Analysis through Novel Feature Engineering and Digital Shadow Modelling: A Case Study in Kiel Fjord}, year = 2024 }
• Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification. Huseljic, Denis; Hahn, Paul; Herde, Marek; Rauch, Lukas; Sick, Bernhard. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), bll 280–296. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Deep active learning (AL) seeks to minimize the annotation costs for training deep neural networks. \\textsc{Bait, a recently proposed AL strategy based on the Fisher Information, has demonstrated impressive performance across various datasets. However, \\textsc{Bait's high computational and memory requirements hinder its applicability on large-scale classification tasks, resulting in current research neglecting \\textsc{Bait in their evaluation. This paper introduces two methods to enhance \\textsc{Bait's computational efficiency and scalability. Notably, we significantly reduce its time complexity by approximating the Fisher Information. In particular, we adapt the original formulation by i) taking the expectation over the most probable classes, and ii) constructing a binary classification task, leading to an alternative likelihood for gradient computations. Consequently, this allows the efficient use of \\textsc{Bait on large-scale datasets, including ImageNet. Our unified and comprehensive evaluation across a variety of datasets demonstrates that our approximations achieve strong performance with considerably reduced time complexity. Furthermore, we provide an extensive open-source toolbox that implements recent state-of-the-art AL strategies, available at \\url{https://github.com/dhuseljic/dal-toolbox.

  @inproceedings{huseljic2024fast, abstract = {Deep active learning (AL) seeks to minimize the annotation costs for training deep neural networks. \\textsc{Bait}, a recently proposed AL strategy based on the Fisher Information, has demonstrated impressive performance across various datasets. However, \\textsc{Bait}'s high computational and memory requirements hinder its applicability on large-scale classification tasks, resulting in current research neglecting \\textsc{Bait} in their evaluation. This paper introduces two methods to enhance \\textsc{Bait}'s computational efficiency and scalability. Notably, we significantly reduce its time complexity by approximating the Fisher Information. In particular, we adapt the original formulation by i) taking the expectation over the most probable classes, and ii) constructing a binary classification task, leading to an alternative likelihood for gradient computations. Consequently, this allows the efficient use of \\textsc{Bait} on large-scale datasets, including ImageNet. Our unified and comprehensive evaluation across a variety of datasets demonstrates that our approximations achieve strong performance with considerably reduced time complexity. Furthermore, we provide an extensive open-source toolbox that implements recent state-of-the-art AL strategies, available at \\url{https://github.com/dhuseljic/dal-toolbox}.}, author = {Huseljic, Denis and Hahn, Paul and Herde, Marek and Rauch, Lukas and Sick, Bernhard}, booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD)}, keywords = {isac-www}, pages = {280–296}, title = {Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification}, year = 2024 }
• Spatial-Temporal Attention Graph Neural Network with Uncertainty Estimation for Remaining Useful Life Prediction. Huang, Zhixin; Nivarthi, Chandana Priya; Gruhl, Christian; Sick, Bernhard. In International Joint Conference on Neural Networks (IJCNN), bll 1–9. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  In the increasingly complex industrial system health management domain, accurate prediction of remaining useful life plays an essential role. This paper analyzes the methods to improve the predictive performance of remaining useful life from three aspects: optimizing model structures, augmenting uncertainty estimation in predictions, and transitioning normalization methods. Based on our analysis, we propose a novel model, the Uncertainty Spatial-Temporal Attention Graph Neural Network (USTAGNN), which consists of three primary components: sensor graph construction, a spatio-temporal feature extractor, and a probabilistic prediction module. The feature extractor leverages graph neural networks and temporal convolutional networks as a foundation to extract spatial and temporal features, further enhanced by attention mechanisms, spectral normalization, and residual connections to bolster its distance awareness. Following extensive experimental comparisons, we utilized the parameter-driven dynamic adjacency matrix for sensor graph construction and the deep kernel Gaussian process for precise uncertainty estimation. USTAGNN tries to resolve issues not thoroughly addressed in existing research, such as comparative analyses of sensor graph construction methods, accurate uncertainty estimation, and the model’s generalization under different preprocessing conditions. The proposed model demonstrated state-of-the-art performance on various subsets of the C-MAPSS dataset, achieving up to a 35.9% improvement in prediction score.

  @inproceedings{huang2024spatial, abstract = {In the increasingly complex industrial system health management domain, accurate prediction of remaining useful life plays an essential role. This paper analyzes the methods to improve the predictive performance of remaining useful life from three aspects: optimizing model structures, augmenting uncertainty estimation in predictions, and transitioning normalization methods. Based on our analysis, we propose a novel model, the Uncertainty Spatial-Temporal Attention Graph Neural Network (USTAGNN), which consists of three primary components: sensor graph construction, a spatio-temporal feature extractor, and a probabilistic prediction module. The feature extractor leverages graph neural networks and temporal convolutional networks as a foundation to extract spatial and temporal features, further enhanced by attention mechanisms, spectral normalization, and residual connections to bolster its distance awareness. Following extensive experimental comparisons, we utilized the parameter-driven dynamic adjacency matrix for sensor graph construction and the deep kernel Gaussian process for precise uncertainty estimation. USTAGNN tries to resolve issues not thoroughly addressed in existing research, such as comparative analyses of sensor graph construction methods, accurate uncertainty estimation, and the model’s generalization under different preprocessing conditions. The proposed model demonstrated state-of-the-art performance on various subsets of the C-MAPSS dataset, achieving up to a 35.9% improvement in prediction score.}, author = {Huang, Zhixin and Nivarthi, Chandana Priya and Gruhl, Christian and Sick, Bernhard}, booktitle = {International Joint Conference on Neural Networks (IJCNN)}, keywords = {Spatio-Temporal-Attention}, pages = {1–9}, publisher = {IEEE}, title = {Spatial-Temporal Attention Graph Neural Network with Uncertainty Estimation for Remaining Useful Life Prediction}, year = 2024 }
• Multi-Task Representation Learning with Temporal Attention for Zero-Shot Time Series Anomaly Detection. Nivarthi, Chandana Priya; Huang, Zhixin; Gruhl, Christian; Sick, Bernhard. In International Joint Conference on Neural Networks (IJCNN), bll 1–10. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Ensuring the reliability of critical industrial systems across various sectors is crucial. It is essential to detect deviations from regular behaviour to mitigate disruptions and preserve infrastructure integrity. However, accurately labelling anomaly datasets is challenging due to their rarity and manual annotation subjectivity. The conventional approach of training separate models for each dataset entity further complicates model development. This paper presents a novel Multi-task Learning framework combining LSTM Autoencoder with temporal attention mechanism (MTL-LATAM) for effective time series anomaly detection. Multitask learning models improve adaptability and generalizability, leading to reduced runtime and compute power while supporting zero-shot evaluation. These models offer flexibility in detecting emerging anomalies. Additionally, we introduce a dynamic thresholding mechanism to incorporate temporal context for anomaly detection and provide visualizations of attention weights to enhance interpretability. The study compares MTL- LATAM, with other multi-task models, evaluates multi-task versus single-task models and assesses the performance of the proposed frame- work in zero-shot learning scenarios. The findings indicate MTL- LATAM’s effectiveness across real-world and open-source datasets, achieving 95% and 97% task synergy. The results underscore the superior performance of multi-task models in zero-shot tasks compared to individual models trained exclusively on their respective datasets.

  @inproceedings{nivarthi2024multi, abstract = {Ensuring the reliability of critical industrial systems across various sectors is crucial. It is essential to detect deviations from regular behaviour to mitigate disruptions and preserve infrastructure integrity. However, accurately labelling anomaly datasets is challenging due to their rarity and manual annotation subjectivity. The conventional approach of training separate models for each dataset entity further complicates model development. This paper presents a novel Multi-task Learning framework combining LSTM Autoencoder with temporal attention mechanism (MTL-LATAM) for effective time series anomaly detection. Multitask learning models improve adaptability and generalizability, leading to reduced runtime and compute power while supporting zero-shot evaluation. These models offer flexibility in detecting emerging anomalies. Additionally, we introduce a dynamic thresholding mechanism to incorporate temporal context for anomaly detection and provide visualizations of attention weights to enhance interpretability. The study compares MTL- LATAM, with other multi-task models, evaluates multi-task versus single-task models and assesses the performance of the proposed frame- work in zero-shot learning scenarios. The findings indicate MTL- LATAM’s effectiveness across real-world and open-source datasets, achieving 95% and 97% task synergy. The results underscore the superior performance of multi-task models in zero-shot tasks compared to individual models trained exclusively on their respective datasets.}, author = {Nivarthi, Chandana Priya and Huang, Zhixin and Gruhl, Christian and Sick, Bernhard}, booktitle = {International Joint Conference on Neural Networks (IJCNN)}, keywords = {isac-www}, pages = {1–10}, publisher = {IEEE}, title = {Multi-Task Representation Learning with Temporal Attention for Zero-Shot Time Series Anomaly Detection}, year = 2024 }
• An Efficient Multi Quantile Regression Network with Ad Hoc Prevention of Quantile Crossing. Decke, Jens; Jenß, Arne; Sick, Bernhard; Gruhl, Christian. In International Conference on Architecture of Computing Systems (ARCS), bll 51–66. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  This article presents the Sorting Composite Quantile Regression Neural Network (SCQRNN), an advanced quantile regression model designed to prevent quantile crossing and enhance computational efficiency. Integrating ad hoc sorting in training, the SCQRNN ensures non-intersecting quantiles, boosting model reliability and interpretability. We demonstrate that the SCQRNN not only prevents quantile crossing and reduces computational complexity but also achieves faster convergence than traditional models. This advancement meets the requirements of high-performance computing for sustainable, accurate computation. In organic computing, the SCQRNN enhances self-aware systems with predictive uncertainties, enriching applications across finance, meteorology, climate science, and engineering.

  @inproceedings{decke2024efficient, abstract = {This article presents the Sorting Composite Quantile Regression Neural Network (SCQRNN), an advanced quantile regression model designed to prevent quantile crossing and enhance computational efficiency. Integrating ad hoc sorting in training, the SCQRNN ensures non-intersecting quantiles, boosting model reliability and interpretability. We demonstrate that the SCQRNN not only prevents quantile crossing and reduces computational complexity but also achieves faster convergence than traditional models. This advancement meets the requirements of high-performance computing for sustainable, accurate computation. In organic computing, the SCQRNN enhances self-aware systems with predictive uncertainties, enriching applications across finance, meteorology, climate science, and engineering.}, author = {Decke, Jens and Jenß, Arne and Sick, Bernhard and Gruhl, Christian}, booktitle = {International Conference on Architecture of Computing Systems (ARCS)}, keywords = {DifferentiableSorting}, pages = {51–66}, publisher = {Springer}, title = {An Efficient Multi Quantile Regression Network with Ad Hoc Prevention of Quantile Crossing}, year = 2024 }
• From Structured to Unstructured: A Comparative Analysis of Computer Vision and Graph Models in solving Mesh-based PDEs. Decke, Jens; Wünsch, Olaf; Sick, Bernhard; Gruhl, Christian. In International Conference on Architecture of Computing Systems (ARCS), bll 82–96. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  This article investigates the application of computer vision and graph-based models in solving mesh-based partial differential equations within high-performance computing environments. Focusing on structured, graded structured, and unstructured meshes, the study compares the performance and computational efficiency of three computer vision-based models against three graph-based models across three datasets. The research aims to identify the most suitable models for different mesh topographies, particularly highlighting the exploration of graded meshes, a less studied area. Results demonstrate that computer vision-based models, notably U-Net, outperform the graph models in prediction performance and efficiency in two (structured and graded) out of three mesh topographies. The study also reveals the unexpected effectiveness of computer vision-based models in handling unstructured meshes, suggesting a potential shift in methodological approaches for data-driven partial differential equation learning. The article underscores deep learning as a viable and potentially sustainable way to enhance traditional high-performance computing methods, advocating for informed model selection based on the topography of the mesh.

  @inproceedings{decke2024structured, abstract = {This article investigates the application of computer vision and graph-based models in solving mesh-based partial differential equations within high-performance computing environments. Focusing on structured, graded structured, and unstructured meshes, the study compares the performance and computational efficiency of three computer vision-based models against three graph-based models across three datasets. The research aims to identify the most suitable models for different mesh topographies, particularly highlighting the exploration of graded meshes, a less studied area. Results demonstrate that computer vision-based models, notably U-Net, outperform the graph models in prediction performance and efficiency in two (structured and graded) out of three mesh topographies. The study also reveals the unexpected effectiveness of computer vision-based models in handling unstructured meshes, suggesting a potential shift in methodological approaches for data-driven partial differential equation learning. The article underscores deep learning as a viable and potentially sustainable way to enhance traditional high-performance computing methods, advocating for informed model selection based on the topography of the mesh.}, author = {Decke, Jens and Wünsch, Olaf and Sick, Bernhard and Gruhl, Christian}, booktitle = {International Conference on Architecture of Computing Systems (ARCS)}, keywords = {MeshTopographies}, pages = {82–96}, publisher = {Springer}, title = {From Structured to Unstructured: A Comparative Analysis of Computer Vision and Graph Models in solving Mesh-based PDEs}, year = 2024 }
• Fault Detection for Satellite Reaction Wheel Based on Zonotopic Set-membership Estimation. Wang, Zhenhua; Xu, Chi; Liu, Zonglin; Shen, Yi. In IEEE Transactions on Aerospace and Electronic Systems, bll 1–12. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  A fault detection method is proposed based on zonotopic set-membership estimation to detect faults in the reaction wheel. First, considering the unknown but bound noise, a zonotope-based estimation is designed based on the nonlinear model of the reaction wheel using the mean value theorem. Then, the residuals and the adaptive threshold are based on the propri zonotopic estimation, used to detect the fault in parameters including the coulomb friction torque coefficient, the bus voltage, and the motor torque coefficient. Finally, the effectiveness of the proposed method is verified by numerical simulations. The simulation results illustrate that the proposed methods can accurately detect different fault scenarios.

  @article{wang2024fault, abstract = {A fault detection method is proposed based on zonotopic set-membership estimation to detect faults in the reaction wheel. First, considering the unknown but bound noise, a zonotope-based estimation is designed based on the nonlinear model of the reaction wheel using the mean value theorem. Then, the residuals and the adaptive threshold are based on the propri zonotopic estimation, used to detect the fault in parameters including the coulomb friction torque coefficient, the bus voltage, and the motor torque coefficient. Finally, the effectiveness of the proposed method is verified by numerical simulations. The simulation results illustrate that the proposed methods can accurately detect different fault scenarios.}, author = {Wang, Zhenhua and Xu, Chi and Liu, Zonglin and Shen, Yi}, journal = {IEEE Transactions on Aerospace and Electronic Systems}, keywords = {isac-www}, pages = {1-12}, title = {Fault Detection for Satellite Reaction Wheel Based on Zonotopic Set-membership Estimation}, year = 2024 }
• Capturing Nonlinear Electron Dynamics with Fully Characterised Attosecond X-ray Pulses. Funke, Lars; Ilchen, Markus; Dingel, Kristina; Mazza, Tommaso; Mullins, Terence; Otto, Thorsten; Rivas, Daniel; Savio, Sara; Serkez, Svitozar; Walter, Peter; Wieland, Niclas; Wülfing, Lasse; Bari, Sadia; Boll, Rebecca; Braune, Markus; Calegari, Francesca; De Fanis, Alberto; Decking, Winfried; Duensing, Andreas; Düsterer, Stefan; Ehresmann, Arno; Erk, Benjamin; Ferreira de Lima, Danilo Enoque; Galler, Andreas; Geloni, Gianluca; Grünert, Jan; Guetg, Marc; Grychtol, Patrik; Hans, Andreas; Held, Arne; Hindriksson, Ruda; Inhester, Ludger; Jahnke, Till; Laksman, Joakim; Larsson, Mats; Liu, Jia; Marangos, Jon P.; Marder, Lutz; Meier, David; Meyer, Michael; Mirian, Najmeh; Ott, Christian; Passow, Christopher; Pfeifer, Thomas; Rupprecht, Patrick; Schletter, Albert; Schmidt, Philipp; Scholz, Frank; Schott, Simon; Schneidmiller, Evgeny; Sick, Bernhard; Son, Sang-Kil; Tiedtke, Kai; Usenko, Sergey; Wanie, Vincent; Wurzer, Markus; Yurkov, Mikhail; Zhaunerchyk, Vitali; Helml, Wolfram. In arXiv e-prints, bl arXiv:2408.03858. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Attosecond X-ray pulses are the key to studying electron dynamics at their natural time scale involving specific electronic states. They are promising to build the conceptual bridge between physical and chemical photo-reaction processes. Free-electron lasers have demonstrated their capability of generating intense attosecond X-ray pulses. However, harnessing them for time-resolving experiments and investigations of nonlinear X-ray absorption mechanisms remains a cutting-edge challenge. We have characterised X-ray pulses with durations of down to 700attoseconds and peak powers up to 200GW at ∼ 1keV photon energy via angular streaking at the SQS instrument of the European XFEL. As direct application, we present results of nonlinear X-ray-matter interaction via state-specific spectroscopy on a transient system. Using the derived spectral and temporal information of each pulse, we deliberately steer the probability for formation of double-core vacancies in neon gas atoms through excitation or ionisation of the second inner-shell electron after K-shell ionisation. Our results advance the field of attosecond science with highly intense and fully characterised X-ray pulses to the site-specific investigation of electronic motion in transient media.

  @article{funke2024capturing, abstract = {Attosecond X-ray pulses are the key to studying electron dynamics at their natural time scale involving specific electronic states. They are promising to build the conceptual bridge between physical and chemical photo-reaction processes. Free-electron lasers have demonstrated their capability of generating intense attosecond X-ray pulses. However, harnessing them for time-resolving experiments and investigations of nonlinear X-ray absorption mechanisms remains a cutting-edge challenge. We have characterised X-ray pulses with durations of down to 700attoseconds and peak powers up to 200GW at ∼ 1keV photon energy via angular streaking at the SQS instrument of the European XFEL. As direct application, we present results of nonlinear X-ray-matter interaction via state-specific spectroscopy on a transient system. Using the derived spectral and temporal information of each pulse, we deliberately steer the probability for formation of double-core vacancies in neon gas atoms through excitation or ionisation of the second inner-shell electron after K-shell ionisation. Our results advance the field of attosecond science with highly intense and fully characterised X-ray pulses to the site-specific investigation of electronic motion in transient media.}, author = {Funke, Lars and Ilchen, Markus and Dingel, Kristina and Mazza, Tommaso and Mullins, Terence and Otto, Thorsten and Rivas, Daniel and Savio, Sara and Serkez, Svitozar and Walter, Peter and Wieland, Niclas and Wülfing, Lasse and Bari, Sadia and Boll, Rebecca and Braune, Markus and Calegari, Francesca and De Fanis, Alberto and Decking, Winfried and Duensing, Andreas and Düsterer, Stefan and Ehresmann, Arno and Erk, Benjamin and Ferreira de Lima, Danilo Enoque and Galler, Andreas and Geloni, Gianluca and Grünert, Jan and Guetg, Marc and Grychtol, Patrik and Hans, Andreas and Held, Arne and Hindriksson, Ruda and Inhester, Ludger and Jahnke, Till and Laksman, Joakim and Larsson, Mats and Liu, Jia and Marangos, Jon P. and Marder, Lutz and Meier, David and Meyer, Michael and Mirian, Najmeh and Ott, Christian and Passow, Christopher and Pfeifer, Thomas and Rupprecht, Patrick and Schletter, Albert and Schmidt, Philipp and Scholz, Frank and Schott, Simon and Schneidmiller, Evgeny and Sick, Bernhard and Son, Sang-Kil and Tiedtke, Kai and Usenko, Sergey and Wanie, Vincent and Wurzer, Markus and Yurkov, Mikhail and Zhaunerchyk, Vitali and Helml, Wolfram}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2408.03858}, title = {Capturing Nonlinear Electron Dynamics with Fully Characterised Attosecond X-ray Pulses}, year = 2024 }
• Hierarchical Motion Planning for Consistent and Safe Decisions in Cooperative Autonomous Driving. Eilbrecht, J.; Stursberg, O. C. Stiller (red.), bll 303–320. Springer International Publishing, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The immersion of autonomous cars in continuously changing environments of on-road traffic requires procedures for decision-making with fast adaptation as well as guarantees on safe motion and collision-avoidance. This contribution proposes a three-layer hierarchic decomposition of the task of automatically steering the autonomous car along a designated route in cooperation with neighbored vehicles. The upper layer of the hierarchy identifies cooperative groups of those vehicles which are involved in a joint scenario for a phase of the planning horizon. The medium layer employs set-based computations of the free space for any vehicle of a joint scenario together with constrained optimal control to determine optimized motion plans. These plans are used on the lower layer as reference signals for tracking control in order to realize motion trajectories. The architecture ensures consistency of the vehicle motion with respect to safety for given assumptions, as well as relatively small computation times by combining offline with online computation.

  @inbook{eilbrecht2024hierarchical, abstract = {The immersion of autonomous cars in continuously changing environments of on-road traffic requires procedures for decision-making with fast adaptation as well as guarantees on safe motion and collision-avoidance. This contribution proposes a three-layer hierarchic decomposition of the task of automatically steering the autonomous car along a designated route in cooperation with neighbored vehicles. The upper layer of the hierarchy identifies cooperative groups of those vehicles which are involved in a joint scenario for a phase of the planning horizon. The medium layer employs set-based computations of the free space for any vehicle of a joint scenario together with constrained optimal control to determine optimized motion plans. These plans are used on the lower layer as reference signals for tracking control in order to realize motion trajectories. The architecture ensures consistency of the vehicle motion with respect to safety for given assumptions, as well as relatively small computation times by combining offline with online computation.}, author = {Eilbrecht, J. and Stursberg, O.}, editor = {Stiller, Christoph}, keywords = {isac-www}, pages = {303-320}, publisher = {Springer International Publishing}, title = {Hierarchical Motion Planning for Consistent and Safe Decisions in Cooperative Autonomous Driving.}, year = 2024 }
• Investigating Reproducibility of Ultra-High Performance Concrete with Consistent Mechanical Properties: A Modeling Pipeline for Sparse Data in Complex Manufacturing. P, Farzad Rezazadeh; Abrishambaf, Amin; Dürrbaum, Axel; Zimmermann, Gregor; Kroll, Andreas. In 34. Workshop Computational Intelligence, bll 143–148. KIT Scientific Publishing, Berlin, Germany, 2024.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{Rezazadeh2024GMA, address = {Berlin, Germany}, author = {P, Farzad Rezazadeh and Abrishambaf, Amin and Dürrbaum, Axel and Zimmermann, Gregor and Kroll, Andreas}, booktitle = {34. Workshop Computational Intelligence}, keywords = {isac-www}, month = {21.–22. November}, pages = {143–148}, publisher = {KIT Scientific Publishing}, title = {Investigating Reproducibility of Ultra-High Performance Concrete with Consistent Mechanical Properties: A Modeling Pipeline for Sparse Data in Complex Manufacturing}, year = 2024 }
• Adaptive Shapley: Using Explainable AI with Large Datasets to Quantify the Impact of Arbitrary Error Sources. Magnussen, Birk Martin; Jessulat, Maik; Stern, Claudius; Sick, Bernhard. In International Conference on Big Data Analytics (ICBDA), bll 305–310. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Almost all sensors suffer from some level of uncertainty introduced from production inaccuracies. When the sensor data is processed by machine learning, quantifying the impact of such production inaccuracies on the output of the machine learning model becomes difficult. Certain neural network architectures, such as continuous feature networks, allow individual features and data to be omitted while still being able to correctly predict the result without the need for retraining. Such features can, for example, be individual channels of a sensor. This article proposes a method to use the capability to omit arbitrary features or sensor channels to calculate Shapley values for each sensor channel. These Shapley values represent the contribution of each individual channel to the measurement. They are defined using an arbitrary function called the ``value function''. If the value function is defined as the error of the current measurement, the Shapley values will represent the contribution of each sensor channel to the error of the measurement result. By calculating Shapley values like this for a large unlabelled dataset of measurements, it is possible to understand how much measurement error was introduced by which channel of which sensor in each measurement. Averaging the Shapley values for each sensor in the dataset will then result in a metric for each channel of that sensor, which represents a contribution to measurement errors. By comparing these values to any arbitrary quality metrics for the sensor channels obtained in a calibration process or similar step, it is possible to correlate and quantify which value in the quality metric will cause how much of a measurement error, or whether the quality metric is even relevant for the measurement accuracy. This article will show the efficacy and use case of the method on an example of the production and quality control of optical sensors based on multiple spatially resolved reflection spectroscopy.

  @inproceedings{magnussen2024adaptive, abstract = {Almost all sensors suffer from some level of uncertainty introduced from production inaccuracies. When the sensor data is processed by machine learning, quantifying the impact of such production inaccuracies on the output of the machine learning model becomes difficult. Certain neural network architectures, such as continuous feature networks, allow individual features and data to be omitted while still being able to correctly predict the result without the need for retraining. Such features can, for example, be individual channels of a sensor. This article proposes a method to use the capability to omit arbitrary features or sensor channels to calculate Shapley values for each sensor channel. These Shapley values represent the contribution of each individual channel to the measurement. They are defined using an arbitrary function called the ``value function''. If the value function is defined as the error of the current measurement, the Shapley values will represent the contribution of each sensor channel to the error of the measurement result. By calculating Shapley values like this for a large unlabelled dataset of measurements, it is possible to understand how much measurement error was introduced by which channel of which sensor in each measurement. Averaging the Shapley values for each sensor in the dataset will then result in a metric for each channel of that sensor, which represents a contribution to measurement errors. By comparing these values to any arbitrary quality metrics for the sensor channels obtained in a calibration process or similar step, it is possible to correlate and quantify which value in the quality metric will cause how much of a measurement error, or whether the quality metric is even relevant for the measurement accuracy. This article will show the efficacy and use case of the method on an example of the production and quality control of optical sensors based on multiple spatially resolved reflection spectroscopy.}, author = {Magnussen, Birk Martin and Jessulat, Maik and Stern, Claudius and Sick, Bernhard}, booktitle = {International Conference on Big Data Analytics (ICBDA)}, keywords = {shapley_values}, pages = {305–310}, publisher = {IEEE}, title = {Adaptive Shapley: Using Explainable AI with Large Datasets to Quantify the Impact of Arbitrary Error Sources}, year = 2024 }
• Tailored Transformations for Constraint Satisfaction in Learning of Parametric Controllers for Nonlinear Systems. Markolf, L.; Stursberg, O. In 10th Int. Conf. on Control, Decision, and Information Technologies. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  The paper on hand addresses the problem of guaranteed satisfaction of ellipsoidal state and input constraints when learning parametric controllers for discrete-time nonlinear but control-affine systems with additive bounded disturbances. The parametric controllers are proposed to be compositions of parametric functions (like neural networks), for which state-dependent transformations lead to ranges that are equal to sets of admissible control inputs derived from offline computed reachable state sets. In this way, constraint satisfaction is guaranteed for all control laws that can be learned through the choice of the parameter vector.

  @article{markolf2024tailored, abstract = {The paper on hand addresses the problem of guaranteed satisfaction of ellipsoidal state and input constraints when learning parametric controllers for discrete-time nonlinear but control-affine systems with additive bounded disturbances. The parametric controllers are proposed to be compositions of parametric functions (like neural networks), for which state-dependent transformations lead to ranges that are equal to sets of admissible control inputs derived from offline computed reachable state sets. In this way, constraint satisfaction is guaranteed for all control laws that can be learned through the choice of the parameter vector.}, author = {Markolf, L. and Stursberg, O.}, journal = {10th Int. Conf. on Control, Decision, and Information Technologies}, keywords = {isac-www}, title = {Tailored Transformations for Constraint Satisfaction in Learning of Parametric Controllers for Nonlinear Systems}, year = 2024 }
• Distributed State Estimation of Linear Systems With Randomly Switching Communication Graphs. Baum, E.; Liu, Z.; Stursberg, O. In IEEE Control Systems Letters, 8, bll 1781–1786. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  This paper introduces two novel methods to synthesize distributed observers for time-varying communication graphs which are modeled as continuous-time Markov processes. The first method obtains all design parameters in a centralized step by solving a system of coupled LMI, while the second method utilizes an observability decomposition to design the parameters locally by partitioning into observable and unobservable system parts. For both methods, the paper provides sufficient conditions for the existence of distributed observers which asymptotically achieve omniscience almost surely. A numerical example to compare the two methods is provided in addition.

  @article{baum2024distributed, abstract = {This paper introduces two novel methods to synthesize distributed observers for time-varying communication graphs which are modeled as continuous-time Markov processes. The first method obtains all design parameters in a centralized step by solving a system of coupled LMI, while the second method utilizes an observability decomposition to design the parameters locally by partitioning into observable and unobservable system parts. For both methods, the paper provides sufficient conditions for the existence of distributed observers which asymptotically achieve omniscience almost surely. A numerical example to compare the two methods is provided in addition.}, author = {Baum, E. and Liu, Z. and Stursberg, O.}, journal = {IEEE Control Systems Letters}, keywords = {isac-www}, pages = {1781-1786}, title = {Distributed State Estimation of Linear Systems With Randomly Switching Communication Graphs}, volume = 8, year = 2024 }
• Approximations of Optimal Control Laws for Constrained Piecewise Affine Systems by Deep Neural Networks. Markolf, L.; Siewert, A.; Stursberg, O. In Proc. of the 2024 European Control Conference, bll 946–952. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  The paper on hand considers the optimal control of piecewise affine systems subject to polytopic constraints. While this problem can be addressed by receding horizon control, the approach is known to be computationally demanding. This paper considers the approximation of receding horizon control laws by deep artificial feed-forward neural networks. The concept of projecting inadmissible inputs onto regions derived from feasible sets is extended to the considered problem setup in order to achieve deterministic guarantees on feasibility and constraint satisfaction. Two approaches are proposed and illustrated in numerical examples.

  @article{markolf2024approximations, abstract = {The paper on hand considers the optimal control of piecewise affine systems subject to polytopic constraints. While this problem can be addressed by receding horizon control, the approach is known to be computationally demanding. This paper considers the approximation of receding horizon control laws by deep artificial feed-forward neural networks. The concept of projecting inadmissible inputs onto regions derived from feasible sets is extended to the considered problem setup in order to achieve deterministic guarantees on feasibility and constraint satisfaction. Two approaches are proposed and illustrated in numerical examples.}, author = {Markolf, L. and Siewert, A. and Stursberg, O.}, journal = {Proc. of the 2024 European Control Conference}, keywords = {isac-www}, pages = {946-952}, title = {Approximations of Optimal Control Laws for Constrained Piecewise Affine Systems by Deep Neural Networks}, year = 2024 }
• Approximation of Limit Cycles by Using Planar Switching Affine Systems with Guarantees for Uniqueness and Stability. Hanke, N.; Liu, Z.; Stursberg, O. In Proc. of the 2024 European Control Conference, bll 1460–1465. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  This paper proposes a novel method to approximate a stable and unique limit cycle, as obtained e.g. frosimulation of a nonlinear model or from experimental data, ba planar switching affine system (PSAS) while preserving thproperties of uniqueness and stability. In contrast to existing literature, which formulates elaborate procedures based on system transformation to conclude on stability and uniqueness, this paper provides easy to check sufficient conditions directly formulated for the PSAS. Those conditions then serve as equality and inequality constraints in an optimization problem to determine the model parameters of the PSAS which best approximate the data points measured for the given limit cycle. It is shown that the sufficient conditions lead to the same results as formulated in literature, while a system transformation is not necessary. Efficiency, flexibility, and performance of the proposed method are demonstrated for a numeric example.

  @article{hanke2024approximation, abstract = {This paper proposes a novel method to approximate a stable and unique limit cycle, as obtained e.g. frosimulation of a nonlinear model or from experimental data, ba planar switching affine system (PSAS) while preserving thproperties of uniqueness and stability. In contrast to existing literature, which formulates elaborate procedures based on system transformation to conclude on stability and uniqueness, this paper provides easy to check sufficient conditions directly formulated for the PSAS. Those conditions then serve as equality and inequality constraints in an optimization problem to determine the model parameters of the PSAS which best approximate the data points measured for the given limit cycle. It is shown that the sufficient conditions lead to the same results as formulated in literature, while a system transformation is not necessary. Efficiency, flexibility, and performance of the proposed method are demonstrated for a numeric example.}, author = {Hanke, N. and Liu, Z. and Stursberg, O.}, journal = {Proc. of the 2024 European Control Conference}, keywords = {isac-www}, pages = {1460-1465}, title = {Approximation of Limit Cycles by Using Planar Switching Affine Systems with Guarantees for Uniqueness and Stability}, year = 2024 }
• Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension. Herde, Marek; Lührs, Lukas; Huseljic, Denis; Sick, Bernhard. In arXiv e-prints, bl arXiv:2405.0338. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that, typically, multiple annotators, e.g., crowdworkers, provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eight state-of-the-art approaches on eleven datasets with noisy class labels provided either by human or simulated annotators. Our code is publicly available through our repository at this https://github.com/ies-research/annot-mix..

  @article{herde2024annot, abstract = {Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that, typically, multiple annotators, e.g., crowdworkers, provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eight state-of-the-art approaches on eleven datasets with noisy class labels provided either by human or simulated annotators. Our code is publicly available through our repository at this https://github.com/ies-research/annot-mix..}, author = {Herde, Marek and Lührs, Lukas and Huseljic, Denis and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2405.0338}, title = {Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension}, year = 2024 }
• Criteria for Uncertainty-based Corner Cases Detection in Instance Segmentation. Heidecker, Florian; El-Khateeb, Ahmad; Bieshaar, Maarten; Sick, Bernhard. In arXiv e-prints, bl arXiv:2404.11266. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The operating environment of a highly automated vehicle is subject to change, e.g., weather, illumination, or the scenario containing different objects and other participants in which the highly automated vehicle has to navigate its passengers safely. These situations must be considered when developing and validating highly automated driving functions. This already poses a problem for training and evaluating deep learning models because without the costly labeling of thousands of recordings, not knowing whether the data contains relevant, interesting data for further model training, it is a guess under which conditions and situations the model performs poorly. For this purpose, we present corner case criteria based on the predictive uncertainty. With our corner case criteria, we are able to detect uncertainty-based corner cases of an object instance segmentation model without relying on ground truth (GT) data. We evaluated each corner case criterion using the COCO and the NuImages dataset to analyze the potential of our approach. We also provide a corner case decision function that allows us to distinguish each object into True Positive (TP), localization and/or classification corner case, or False Positive (FP). We also present our first results of an iterative training cycle that outperforms the baseline and where the data added to the training dataset is selected based on the corner case decision function.

  @article{heidecker2024criteria, abstract = {The operating environment of a highly automated vehicle is subject to change, e.g., weather, illumination, or the scenario containing different objects and other participants in which the highly automated vehicle has to navigate its passengers safely. These situations must be considered when developing and validating highly automated driving functions. This already poses a problem for training and evaluating deep learning models because without the costly labeling of thousands of recordings, not knowing whether the data contains relevant, interesting data for further model training, it is a guess under which conditions and situations the model performs poorly. For this purpose, we present corner case criteria based on the predictive uncertainty. With our corner case criteria, we are able to detect uncertainty-based corner cases of an object instance segmentation model without relying on ground truth (GT) data. We evaluated each corner case criterion using the COCO and the NuImages dataset to analyze the potential of our approach. We also provide a corner case decision function that allows us to distinguish each object into True Positive (TP), localization and/or classification corner case, or False Positive (FP). We also present our first results of an iterative training cycle that outperforms the baseline and where the data added to the training dataset is selected based on the corner case decision function.}, author = {Heidecker, Florian and El-Khateeb, Ahmad and Bieshaar, Maarten and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2404.11266}, title = {Criteria for Uncertainty-based Corner Cases Detection in Instance Segmentation}, year = 2024 }
• Organic Computing -- Doctoral Dissertation Colloquium 2023. Tomforde, Sven; Krupitzer, Christian. In Vol. 26Intelligent Embedded Systems. kassel university press, 2024.
  • [ BibTeX ]
  @book{tomforde2024organic, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {isac-www}, publisher = {kassel university press}, series = {Intelligent Embedded Systems}, title = {Organic Computing – Doctoral Dissertation Colloquium 2023}, volume = 26, year = 2024 }
• Enhancing Multi-Objective Optimization through Machine Learning-Supported Multiphysics Simulation. Botache, Diego; Decke, Jens; Ripken, Winfried; Dornipati, Abhinay; Götz-Hahn, Franz; Ayeb, Mohamed; Sick, Bernhard. In arXiv e-prints, bl arXiv:2309.13179v2. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper presents a methodological framework for training, self-optimising, and self-organising surrogate models to approximate and speed up multiobjective optimisation of technical systems based on multiphysics simulations. At the hand of two real-world datasets, we illustrate that surrogate models can be trained on relatively small amounts of data to approximate the underlying simulations accurately. Including explainable AI techniques allow for highlighting feature relevancy or dependencies and supporting the possible extension of the used datasets. One of the datasets was created for this paper and is made publicly available for the broader scientific community. Extensive experiments combine four machine learning and deep learning algorithms with an evolutionary optimisation algorithm. The performance of the combined training and optimisation pipeline is evaluated by verifying the generated Pareto-optimal results using the ground truth simulations. The results from our pipeline and a comprehensive evaluation strategy show the potential for efficiently acquiring solution candidates in multiobjective optimisation tasks by reducing the number of simulations and conserving a higher prediction accuracy, i.e., with a MAPE score under 5% for one of the presented use cases.

  @article{botache2024enhancing, abstract = {This paper presents a methodological framework for training, self-optimising, and self-organising surrogate models to approximate and speed up multiobjective optimisation of technical systems based on multiphysics simulations. At the hand of two real-world datasets, we illustrate that surrogate models can be trained on relatively small amounts of data to approximate the underlying simulations accurately. Including explainable AI techniques allow for highlighting feature relevancy or dependencies and supporting the possible extension of the used datasets. One of the datasets was created for this paper and is made publicly available for the broader scientific community. Extensive experiments combine four machine learning and deep learning algorithms with an evolutionary optimisation algorithm. The performance of the combined training and optimisation pipeline is evaluated by verifying the generated Pareto-optimal results using the ground truth simulations. The results from our pipeline and a comprehensive evaluation strategy show the potential for efficiently acquiring solution candidates in multiobjective optimisation tasks by reducing the number of simulations and conserving a higher prediction accuracy, i.e., with a MAPE score under 5% for one of the presented use cases.}, author = {Botache, Diego and Decke, Jens and Ripken, Winfried and Dornipati, Abhinay and Götz-Hahn, Franz and Ayeb, Mohamed and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {Explainable_Artificial_Intelligence}, pages = {arXiv:2309.13179v2}, title = {Enhancing Multi-Objective Optimization through Machine Learning-Supported Multiphysics Simulation}, year = 2024 }
• Weisfeiler–Lehman goes dynamic: An analysis of the expressive power of Graph Neural Networks for attributed and dynamic graphs. Beddar-Wiesing, Silvia; D’Inverno, Alessio; Graziani, Caterina; Lachi, Veronica; Moallemy-Oureh, Alice; Scarselli, Franco; Thomas, Josephine. In Neural Networks, 173, bl 106213. Elsevier, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Graph Neural Networks (GNNs) are a large class of relational models for graph processing. Recent theoretical studies on the expressive power of GNNs have focused on two issues. On the one hand, it has been proven that GNNs are as powerful as the Weisfeiler–Lehman test (1-WL) in their ability to distinguish graphs. Moreover, it has been shown that the equivalence enforced by 1-WL equals unfolding equivalence. On the other hand, GNNs turned out to be universal approximators on graphs modulo the constraints enforced by 1-WL/unfolding equivalence. However, these results only apply to Static Attributed Undirected Homogeneous Graphs (SAUHG) with node attributes. In contrast, real-life applications often involve a much larger variety of graph types. In this paper, we conduct a theoretical analysis of the expressive power of GNNs for two other graph domains that are particularly interesting in practical applications, namely dynamic graphs and SAUGHs with edge attributes. Dynamic graphs are widely used in modern applications; hence, the study of the expressive capability of GNNs in this domain is essential for practical reasons and, in addition, it requires a new analyzing approach due to the difference in the architecture of dynamic GNNs compared to static ones. On the other hand, the examination of SAUHGs is of particular relevance since they act as a standard form for all graph types: it has been shown that all graph types can be transformed without loss of information to SAUHGs with both attributes on nodes and edges. This paper considers generic GNN models and appropriate 1-WL tests for those domains. Then, the known results on the expressive power of GNNs are extended to the mentioned domains: it is proven that GNNs have the same capability as the 1-WL test, the 1-WL equivalence equals unfolding equivalence and that GNNs are universal approximators modulo 1-WL/unfolding equivalence. Moreover, the proof of the approximation capability is mostly constructive and allows us to deduce hints on the architecture of GNNs that can achieve the desired approximation.

  @article{beddarwiesing2024weisfeiler, abstract = {Graph Neural Networks (GNNs) are a large class of relational models for graph processing. Recent theoretical studies on the expressive power of GNNs have focused on two issues. On the one hand, it has been proven that GNNs are as powerful as the Weisfeiler–Lehman test (1-WL) in their ability to distinguish graphs. Moreover, it has been shown that the equivalence enforced by 1-WL equals unfolding equivalence. On the other hand, GNNs turned out to be universal approximators on graphs modulo the constraints enforced by 1-WL/unfolding equivalence. However, these results only apply to Static Attributed Undirected Homogeneous Graphs (SAUHG) with node attributes. In contrast, real-life applications often involve a much larger variety of graph types. In this paper, we conduct a theoretical analysis of the expressive power of GNNs for two other graph domains that are particularly interesting in practical applications, namely dynamic graphs and SAUGHs with edge attributes. Dynamic graphs are widely used in modern applications; hence, the study of the expressive capability of GNNs in this domain is essential for practical reasons and, in addition, it requires a new analyzing approach due to the difference in the architecture of dynamic GNNs compared to static ones. On the other hand, the examination of SAUHGs is of particular relevance since they act as a standard form for all graph types: it has been shown that all graph types can be transformed without loss of information to SAUHGs with both attributes on nodes and edges. This paper considers generic GNN models and appropriate 1-WL tests for those domains. Then, the known results on the expressive power of GNNs are extended to the mentioned domains: it is proven that GNNs have the same capability as the 1-WL test, the 1-WL equivalence equals unfolding equivalence and that GNNs are universal approximators modulo 1-WL/unfolding equivalence. Moreover, the proof of the approximation capability is mostly constructive and allows us to deduce hints on the architecture of GNNs that can achieve the desired approximation.}, author = {Beddar-Wiesing, Silvia and D'Inverno, Alessio and Graziani, Caterina and Lachi, Veronica and Moallemy-Oureh, Alice and Scarselli, Franco and Thomas, Josephine}, journal = {Neural Networks}, keywords = {test}, pages = 106213, publisher = {Elsevier}, title = {Weisfeiler–Lehman goes dynamic: An analysis of the expressive power of Graph Neural Networks for attributed and dynamic graphs}, volume = 173, year = 2024 }
• Optical Detection of the Body Mass Index and Related Parameters Using Multiple Spatially Resolved Reflection Spectroscopy. Magnussen, Birk Martin; Möckel, Frank; Jessulat, Maik; Stern, Claudius; Sick, Bernhard. In International Conference on Bioinformatics and Computational Biology (ICBCB). IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Obesity is a common problem for many people. In order to assist people combating obesity, providing methods to quickly, easily, and inexpensively assess their body composition is important. This article investigates how noninvasive, optical sensors based on multiple spatially resolved reflection spectroscopy can be used to measure the body mass index and body composition parameters. Using machine learning to train continuous feature networks, it is possible to predict the body mass index of a subject, with a correlation of \($R=0.61$\) with \($p<0.0001$\). Similarly, the predicted body mass index shows correlations to both the subject's visceral fat (\($R=0.44,p=0.0023$\)) and skeletal muscle mass index (\($R=0.52,p=0.0003$\)), indicating that the trained neural network is capable of identifying both types of tissue. Strategies to independently detect either type of tissue are discussed.

  @inproceedings{magnussen2024optical, abstract = {Obesity is a common problem for many people. In order to assist people combating obesity, providing methods to quickly, easily, and inexpensively assess their body composition is important. This article investigates how noninvasive, optical sensors based on multiple spatially resolved reflection spectroscopy can be used to measure the body mass index and body composition parameters. Using machine learning to train continuous feature networks, it is possible to predict the body mass index of a subject, with a correlation of $R=0.61$ with $p<0.0001$. Similarly, the predicted body mass index shows correlations to both the subject's visceral fat ($R=0.44,p=0.0023$) and skeletal muscle mass index ($R=0.52,p=0.0003$), indicating that the trained neural network is capable of identifying both types of tissue. Strategies to independently detect either type of tissue are discussed.}, author = {Magnussen, Birk Martin and Möckel, Frank and Jessulat, Maik and Stern, Claudius and Sick, Bernhard}, booktitle = {International Conference on Bioinformatics and Computational Biology (ICBCB)}, keywords = {body_mass_index}, note = {(accepted)}, publisher = {IEEE}, title = {Optical Detection of the Body Mass Index and Related Parameters Using Multiple Spatially Resolved Reflection Spectroscopy}, year = 2024 }
• Computing Bounds for the Synchronization Errors of Nonidentical Nonlinear Oscillators with Time-Varying Diffusive Coupling. Trummel, Tabea; Liu, Zonglin; Stursberg, Olaf. In Proc. 21st Int. Conf. on Informatics in Control, Automation and Robotics, (Vol. 1), bll 612–622. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  The paper studies bounds of the synchronization error for networks of diffusively coupled and nonidentical nonlinear oscillators. In contrast to preceding work, which only analyzes the synchronization error in the limit and for constant coupling, a method based on over-approximating reachable sets of the synchronization error is proposed. The method allows the evaluation of different and time-varying gains along the limit cycles. Instead of using strong coupling gains to preserve synchronization, it is proposed to vary the coupling gains over time, while the synchronization error determined via reachable set does not exceed a specified bound. Evaluation results for the proposed method are provided for different types of coupled oscillators.

  @article{noauthororeditor, abstract = {The paper studies bounds of the synchronization error for networks of diffusively coupled and nonidentical nonlinear oscillators. In contrast to preceding work, which only analyzes the synchronization error in the limit and for constant coupling, a method based on over-approximating reachable sets of the synchronization error is proposed. The method allows the evaluation of different and time-varying gains along the limit cycles. Instead of using strong coupling gains to preserve synchronization, it is proposed to vary the coupling gains over time, while the synchronization error determined via reachable set does not exceed a specified bound. Evaluation results for the proposed method are provided for different types of coupled oscillators.}, author = {Trummel, Tabea and Liu, Zonglin and Stursberg, Olaf}, journal = {Proc. 21st Int. Conf. on Informatics in Control, Automation and Robotics}, keywords = {isac-www}, number = {Vol. 1}, pages = {612-622}, title = {Computing Bounds for the Synchronization Errors of Nonidentical Nonlinear Oscillators with Time-Varying Diffusive Coupling}, year = 2024 }
• Data-driven prediction of the surface layer condition in hard-turning for controlling component quality. Wittich, Felix; Wegener, Thomas; Liehr, Alexander; Zinn, Wolfgagn; Niendorf, Thomas; Kroll, Andreas. In Production Engineering, 18, bll 381–392. 2024.
  • [ BibTeX ]
  • [ URL ]
  @article{WittichProdEng2023, author = {Wittich, Felix and Wegener, Thomas and Liehr, Alexander and Zinn, Wolfgagn and Niendorf, Thomas and Kroll, Andreas}, journal = {Production Engineering}, keywords = {isac-www}, pages = {381–392}, title = {Data-driven prediction of the surface layer condition in hard-turning for controlling component quality}, volume = 18, year = 2024 }
• System Level Control and Optimisation of Microgrids. Parisio, Alessandra; Schiffer, Johannes; Hans, Christian A. The Institution of Engineering and Technology, 2024.
  • [ BibTeX ]
  • [ URL ]
  @book{doi:10.1049/PBPO149E, author = {Parisio, Alessandra and Schiffer, Johannes and Hans, Christian A.}, keywords = {isac-www}, publisher = {The Institution of Engineering and Technology}, title = {System Level Control and Optimisation of Microgrids}, year = 2024 }
• Model predictive control of wakes for wind farm power tracking. Sterle, Arnold; Hans, Christian A.; Raisch, Jörg. In Journal of Physics: Conference Series, 2767(3), bl 032005. IOP Publishing, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper, a model predictive control scheme for wind farms is presented. Our approach considers wake dynamics including their influence on local wind conditions and allows the tracking of a given power reference. In detail, a Gaussian wake model is used in combination with observation points that carry wind condition information. This allows the estimation of the rotor effective wind speeds at downstream turbines, based on which we deduce their power output. Through different approximation methods, the associated finite horizon nonlinear optimization problem is reformulated in a mixed-integer quadratically-constrained quadratic program fashion. By solving the reformulated problem online, optimal yaw angles and axial induction factors are found. Closed-loop simulations indicate good power tracking capabilities over a wide range of power setpoints while distributing wind turbine infeed evenly among all units. Additionally, the simulation results underline real time capabilities of our approach.

  @article{Sterle_2024, abstract = {In this paper, a model predictive control scheme for wind farms is presented. Our approach considers wake dynamics including their influence on local wind conditions and allows the tracking of a given power reference. In detail, a Gaussian wake model is used in combination with observation points that carry wind condition information. This allows the estimation of the rotor effective wind speeds at downstream turbines, based on which we deduce their power output. Through different approximation methods, the associated finite horizon nonlinear optimization problem is reformulated in a mixed-integer quadratically-constrained quadratic program fashion. By solving the reformulated problem online, optimal yaw angles and axial induction factors are found. Closed-loop simulations indicate good power tracking capabilities over a wide range of power setpoints while distributing wind turbine infeed evenly among all units. Additionally, the simulation results underline real time capabilities of our approach.}, author = {Sterle, Arnold and Hans, Christian A. and Raisch, Jörg}, journal = {Journal of Physics: Conference Series}, keywords = {isac-www}, month = {jun}, number = 3, pages = {032005}, publisher = {IOP Publishing}, title = {Model predictive control of wakes for wind farm power tracking}, volume = 2767, year = 2024 }
• Identification of Power Systems with Droop-Controlled Units Using Neural Ordinary Differential Equations. Wolf, Hannes M. H.; Hans, Christian A. 2024.
  • [ BibTeX ]
  • [ URL ]
  @misc{wolf2024identificationpowersystemsdroopcontrolled, author = {Wolf, Hannes M. H. and Hans, Christian A.}, keywords = {isac-www}, title = {Identification of Power Systems with Droop-Controlled Units Using Neural Ordinary Differential Equations}, year = 2024 }
• Predictions and Decision Making for Resilient Intelligent Sustainable Energy Systems. Braun, Martin; Gruhl, Christian; Hans, Christian A.; Härtel, Philipp; Scholz, Christoph; Sick, Bernhard; Siefert, Malte; Steinke, Florian; Stursberg, Olaf; von Berg, Sebastian Wende. 2024.
  • [ BibTeX ]
  • [ URL ]
  @misc{braun2024predictionsdecisionmakingresilient, author = {Braun, Martin and Gruhl, Christian and Hans, Christian A. and Härtel, Philipp and Scholz, Christoph and Sick, Bernhard and Siefert, Malte and Steinke, Florian and Stursberg, Olaf and von Berg, Sebastian Wende}, keywords = {isac-www}, title = {Predictions and Decision Making for Resilient Intelligent Sustainable Energy Systems}, year = 2024 }
• Robust LQ Optimal Control for Wind Turbine Power Tracking Operation. Grapentin, Aaron; Hans, Christian A.; Raisch, Jörg. 2024.
  • [ BibTeX ]
  • [ URL ]
  @misc{grapentin2024robustlqoptimalcontrol, author = {Grapentin, Aaron and Hans, Christian A. and Raisch, Jörg}, keywords = {isac-www}, title = {Robust LQ Optimal Control for Wind Turbine Power Tracking Operation}, year = 2024 }
• dopanim: A Dataset of Doppelganger Animals with Noisy Annotations from Multiple Humans. Herde, Marek; Huseljic, Denis; Rauch, Lukas; Sick, Bernhard. In Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks, bll 1–16. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Human annotators typically provide annotated data for training machine learning models, such as neural networks. Yet, human annotations are subject to noise, impairing generalization performances. Methodological research on approaches counteracting noisy annotations requires corresponding datasets for a meaningful empirical evaluation. Consequently, we introduce a novel benchmark dataset, dopanim, consisting of about 15,750 animal images of 15 classes with ground truth labels. For approximately 10,500 of these images, 20 humans provided over 52,000 annotations with an accuracy of circa 67%. Its key attributes include (1) the challenging task of classifying doppelganger animals, (2) human-estimated likelihoods as annotations, and (3) annotator metadata. We benchmark well-known multi-annotator learning approaches using seven variants of this dataset and outline further evaluation use cases such as learning beyond hard class labels and active learning. Our dataset and a comprehensive codebase are publicly available to emulate the data collection process and to reproduce all empirical results.

  @inproceedings{herde2024dopanim, abstract = {Human annotators typically provide annotated data for training machine learning models, such as neural networks. Yet, human annotations are subject to noise, impairing generalization performances. Methodological research on approaches counteracting noisy annotations requires corresponding datasets for a meaningful empirical evaluation. Consequently, we introduce a novel benchmark dataset, dopanim, consisting of about 15,750 animal images of 15 classes with ground truth labels. For approximately 10,500 of these images, 20 humans provided over 52,000 annotations with an accuracy of circa 67%. Its key attributes include (1) the challenging task of classifying doppelganger animals, (2) human-estimated likelihoods as annotations, and (3) annotator metadata. We benchmark well-known multi-annotator learning approaches using seven variants of this dataset and outline further evaluation use cases such as learning beyond hard class labels and active learning. Our dataset and a comprehensive codebase are publicly available to emulate the data collection process and to reproduce all empirical results.}, author = {Herde, Marek and Huseljic, Denis and Rauch, Lukas and Sick, Bernhard}, booktitle = {Conference on Neural Information Processing Systems (NeurIPS 2024) Track on Datasets and Benchmarks}, keywords = {isac-www}, pages = {1–16}, title = {dopanim: A Dataset of Doppelganger Animals with Noisy Annotations from Multiple Humans}, year = 2024 }
• Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension. Herde, Marek; Lührs, Lukas; Huseljic, Denis; Sick, Bernhard. In European Conference on Artificial Intelligence (ECAI), bll 2910–2918. IOS Press, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that multiple annotators, e.g., crowdworkers, typically provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eleven (mostly state-of-the-art) approaches in an evaluation study with eleven datasets comprising noisy class labels from either human or simulated annotators. Our code is publicly available through our GitHub repository at https://github.com/ies-research/multi-annotator-machine-learning/tree/annot-mix.

  @inproceedings{herde2024annota, abstract = {Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that multiple annotators, e.g., crowdworkers, typically provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eleven (mostly state-of-the-art) approaches in an evaluation study with eleven datasets comprising noisy class labels from either human or simulated annotators. Our code is publicly available through our GitHub repository at https://github.com/ies-research/multi-annotator-machine-learning/tree/annot-mix.}, author = {Herde, Marek and Lührs, Lukas and Huseljic, Denis and Sick, Bernhard}, booktitle = {European Conference on Artificial Intelligence (ECAI)}, keywords = {isac-www}, pages = {2910–2918}, publisher = {IOS Press}, title = {Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension}, year = 2024 }
• Exploring 3D Object Detection for Autonomous Factory Driving: Advanced Research on Handling Limited Annotations with Ground Truth Sampling Augmentation. Reuse, Matthias; Amende, Karl; Simon, Martin; Sick, Bernhard. In Computer Sciences & Mathematics Forum, 9(1), bl 5. MDPI, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Autonomously driving vehicles in car factories and parking spaces can represent a competitive advantage in the logistics industry. However, the real-world application is challenging in many ways. First of all, there are no publicly available datasets for this specific task. Therefore, we equipped two industrial production sites with up to 11 LiDAR sensors to collect and annotate our own data for infrastructural 3D object detection. These form the basis for extensive experiments. Due to the still limited amount of labeled data, the commonly used ground truth sampling augmentation is the core of research in this work. Several variations of this augmentation method are explored, revealing that in our case, the most commonly used is not necessarily the best. We show that an easy-to-create polygon can noticeably improve the detection results in this application scenario. By using these augmentation methods, it is even possible to achieve moderate detection results when only empty frames without any objects and a database with only a few labeled objects are used.

  @article{reuse2024exploring, abstract = {Autonomously driving vehicles in car factories and parking spaces can represent a competitive advantage in the logistics industry. However, the real-world application is challenging in many ways. First of all, there are no publicly available datasets for this specific task. Therefore, we equipped two industrial production sites with up to 11 LiDAR sensors to collect and annotate our own data for infrastructural 3D object detection. These form the basis for extensive experiments. Due to the still limited amount of labeled data, the commonly used ground truth sampling augmentation is the core of research in this work. Several variations of this augmentation method are explored, revealing that in our case, the most commonly used is not necessarily the best. We show that an easy-to-create polygon can noticeably improve the detection results in this application scenario. By using these augmentation methods, it is even possible to achieve moderate detection results when only empty frames without any objects and a database with only a few labeled objects are used.}, author = {Reuse, Matthias and Amende, Karl and Simon, Martin and Sick, Bernhard}, journal = {Computer Sciences \& Mathematics Forum}, keywords = {isac-www}, number = 1, pages = 5, publisher = {MDPI}, title = {Exploring 3D Object Detection for Autonomous Factory Driving: Advanced Research on Handling Limited Annotations with Ground Truth Sampling Augmentation}, volume = 9, year = 2024 }
• It is the habit not the handle that affects tooth brushing - a randomised counterbalanced cross over study with young and healthy adults. Deinzer, Renate; Eidenhardt, Zdenka; Sohrabi, Keywan; Stenger, Manuel; Kraft, Dominik; Sick, Bernhard; Götz-Hahn, Franz; Bottenbruch, Carlotta; Berneburg, Nils; Weik, Ulrike. In BMC Oral Health, 24(1), bl 757. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Objective To assess the effect of the toothbrush handle on video-observed toothbrushing behaviour and toothbrushing effectiveness. Methods This is a randomized counterbalanced cross-over study. N = 50 university students and employees brushed their teeth at two occasions, one week apart, using either a commercial ergonomically designed manual toothbrush (MT) or Brushalyze V1 (BV1), a manual toothbrush with a thick cylindrical handle without any specific ergonomic features. Brushing behaviour was video-analysed. Plaque was assessed at the second occasion immediately after brushing. Participants also rated their self-perceived oral cleanliness and directly compared the two brushes regarding their handling and compared them to the brushed they used at home. Results The study participants found the BV1 significantly more cumbersome than the M1 or their brush at home. (p < 0.05). However, correlation analyses revealed a strong consistency of brushing behavior with the two brushes (0.71 < r < 0.91). Means differed only slightly (all d < 0.36). These differences became statistically significant only for the brushing time at inner surfaces (d = 0.31 p = 0.03) and horizontal movements at inner surfaces (d = 0.35, p = 0.02). Plaque levels at the gingival margins did not differ while slightly more plaque persisted at the more coronal aspects of the crown after brushing with BV1 (d = 0.592; p 0.042). Discussion The results of the study indicate that the brushing handle does not play a major role in brushing behavior or brushing effectiveness.

  @article{deinzer2024it, abstract = {Objective To assess the effect of the toothbrush handle on video-observed toothbrushing behaviour and toothbrushing effectiveness. Methods This is a randomized counterbalanced cross-over study. N = 50 university students and employees brushed their teeth at two occasions, one week apart, using either a commercial ergonomically designed manual toothbrush (MT) or Brushalyze V1 (BV1), a manual toothbrush with a thick cylindrical handle without any specific ergonomic features. Brushing behaviour was video-analysed. Plaque was assessed at the second occasion immediately after brushing. Participants also rated their self-perceived oral cleanliness and directly compared the two brushes regarding their handling and compared them to the brushed they used at home. Results The study participants found the BV1 significantly more cumbersome than the M1 or their brush at home. (p < 0.05). However, correlation analyses revealed a strong consistency of brushing behavior with the two brushes (0.71 < r < 0.91). Means differed only slightly (all d < 0.36). These differences became statistically significant only for the brushing time at inner surfaces (d = 0.31 p = 0.03) and horizontal movements at inner surfaces (d = 0.35, p = 0.02). Plaque levels at the gingival margins did not differ while slightly more plaque persisted at the more coronal aspects of the crown after brushing with BV1 (d = 0.592; p 0.042). Discussion The results of the study indicate that the brushing handle does not play a major role in brushing behavior or brushing effectiveness.}, author = {Deinzer, Renate and Eidenhardt, Zdenka and Sohrabi, Keywan and Stenger, Manuel and Kraft, Dominik and Sick, Bernhard and Götz-Hahn, Franz and Bottenbruch, Carlotta and Berneburg, Nils and Weik, Ulrike}, journal = {BMC Oral Health}, keywords = {isac-www}, number = 1, pages = 757, publisher = {Springer}, title = {It is the habit not the handle that affects tooth brushing - a randomised counterbalanced cross over study with young and healthy adults}, volume = 24, year = 2024 }
• Workshop on Interactive Adaptive Learning (IAL), ECML PKDD. Bunse, Mirko; Herde, Marek; Krempl, Georg; Lemaire, Vincent; Tharwat, Alaa; Pham, Minh Tuan; Saadallah, Amal. In Vol. 3770CEUR Workshop Proceedings. 2024.
  • [ BibTeX ]
  • [ URL ]
  @proceedings{bunse2024workshop, editor = {Bunse, Mirko and Herde, Marek and Krempl, Georg and Lemaire, Vincent and Tharwat, Alaa and Pham, Minh Tuan and Saadallah, Amal}, keywords = {isac-www}, series = {{CEUR} Workshop Proceedings}, title = {Workshop on Interactive Adaptive Learning (IAL), ECML PKDD}, volume = 3770, year = 2024 }
• Sensor Equivariance: A Framework for Semantic Segmentation with Diverse Camera Models. Reichert, Hannes; Hetzel, Manuel; Hubert, Andreas; Doll, Konrad; Sick, Bernhard. In Workshop on Omnidirectional Computer Vision (OmniCV), ICVPR, bll 1254–1261. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Objects are represented differently in projection-based sensors such as cameras depending on sensor resolution, field of view, and distortion, leading to distorted physical and geometric properties. As a result, sensor data process- ing depend on these properties. With the large variations of sensors on the market, an equivariant representation and suitable processing are necessary to become independent of the sensor used. In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. Further- more, we introduce a SensorConv layer as an extension to the conventional convolution layer. SensorConv enable us- ing projection properties in convolutional neural networks. To that end, we propose an architecture for using the Sen- sorConv layer in the Detectron2 [21] framework. We col- lected a dataset of equirectangular images for our experi- ments with the CARLA [3] simulator. To analyze multiple sensor models (i.e., sensor intrinsic), we created an aug- mentation method to emulate a high variability of sensors from the collected equirectangular panoramas. In our ex- periment, we show that our method can generalize better across different camera sensors.

  @inproceedings{reichert2024sensor, abstract = {Objects are represented differently in projection-based sensors such as cameras depending on sensor resolution, field of view, and distortion, leading to distorted physical and geometric properties. As a result, sensor data process- ing depend on these properties. With the large variations of sensors on the market, an equivariant representation and suitable processing are necessary to become independent of the sensor used. In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. Further- more, we introduce a SensorConv layer as an extension to the conventional convolution layer. SensorConv enable us- ing projection properties in convolutional neural networks. To that end, we propose an architecture for using the Sen- sorConv layer in the Detectron2 [21] framework. We col- lected a dataset of equirectangular images for our experi- ments with the CARLA [3] simulator. To analyze multiple sensor models (i.e., sensor intrinsic), we created an aug- mentation method to emulate a high variability of sensors from the collected equirectangular panoramas. In our ex- periment, we show that our method can generalize better across different camera sensors.}, author = {Reichert, Hannes and Hetzel, Manuel and Hubert, Andreas and Doll, Konrad and Sick, Bernhard}, booktitle = {Workshop on Omnidirectional Computer Vision (OmniCV), ICVPR}, keywords = {isac-www}, pages = {1254–1261}, publisher = {IEEE}, title = {Sensor Equivariance: A Framework for Semantic Segmentation with Diverse Camera Models}, year = 2024 }
• General Reusability: Ensuring Long-Term Benefits of Deep Active Learning. Hahn, Paul; Huseljic, Denis; Herde, Marek; Sick, Bernhard. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 33–46. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Deep active learning (DAL) provides a promising framework for reducing labeling costs for data-hungry deep neural networks. In DAL research, experiments are evaluated based on the performance of the model that is responsible for querying data, which can be interpreted as a model-centric evaluation. However, the data queried by DAL should be model agnostic so that new models can be effectively trained on it. An example can be derived from the rapid progress in deep learning, where new architectures are proposed in quick succession. Naturally, we want to employ these new and more effective models while still using the queried data from DAL. This results in the need to ensure high performance for both current and future models, which we refer to as reusability. In line with this topic, we propose general reusability, a novel, data-centric evaluation metric that measures the long-term value of queried data. Our experiments demonstrate that the current evaluation protocol in DAL research does not assess the reusability of data and that GR can fill this gap.

  @inproceedings{hahn2024general, abstract = {Deep active learning (DAL) provides a promising framework for reducing labeling costs for data-hungry deep neural networks. In DAL research, experiments are evaluated based on the performance of the model that is responsible for querying data, which can be interpreted as a model-centric evaluation. However, the data queried by DAL should be model agnostic so that new models can be effectively trained on it. An example can be derived from the rapid progress in deep learning, where new architectures are proposed in quick succession. Naturally, we want to employ these new and more effective models while still using the queried data from DAL. This results in the need to ensure high performance for both current and future models, which we refer to as reusability. In line with this topic, we propose general reusability, a novel, data-centric evaluation metric that measures the long-term value of queried data. Our experiments demonstrate that the current evaluation protocol in DAL research does not assess the reusability of data and that GR can fill this gap.}, author = {Hahn, Paul and Huseljic, Denis and Herde, Marek and Sick, Bernhard}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {isac-www}, pages = {33–46}, title = {General Reusability: Ensuring Long-Term Benefits of Deep Active Learning}, year = 2024 }
• Tutorial: Interactive Adaptive Learning. Herde, Marek; Pham, Minh Tuan; Tharwat, Alaa; Sick, Bernhard. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 1–6. 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  We summarize the contents of the tutorial we present as a part of the 8th Interactive Adaptive Learning workshop. This workshop is co-located with the ECML-PKDD conference, where it takes place on September 9th, 2024 in Vilnius, Lithuania.

  @inproceedings{herde2024tutorial, abstract = {We summarize the contents of the tutorial we present as a part of the 8th Interactive Adaptive Learning workshop. This workshop is co-located with the ECML-PKDD conference, where it takes place on September 9th, 2024 in Vilnius, Lithuania.}, author = {Herde, Marek and Pham, Minh Tuan and Tharwat, Alaa and Sick, Bernhard}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {isac-www}, pages = {1–6}, title = {Tutorial: Interactive Adaptive Learning}, year = 2024 }
• Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving. Zernetsch, Stefan; Kress, Viktor; Bieshaar, Maarten; Schneegans, Jan; Reitberger, Günther; Fuchs, Erich; Sick, Bernhard; Doll, Konrad. In Cooperatively Interacting Vehicles: Methods and Effects of Automated Cooperation in Traffic, C. Stiller, M. Althoff, C. Burger, B. Deml, L. Eckstein, F. Flemisch (reds.), bll 35–87. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  The project Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving (DeCoInt{\\($}{\$\)}^2{\\($}{\$\)}2) focuses on detecting the intentions of vulnerable road users (VRUs) in automated driving using cooperative technologies. Especially in urban areas, VRUs, e.g., pedestrians and cyclists, will continue to play an essential role in mixed traffic. For an accident-free and highly efficient traffic flow with automated vehicles, it is vital to perceive VRUs and their intentions and analyze them similarly to humans when driving and forecasting VRU trajectories. Doing this reliably and robustly with a multimodal sensor system (e.g., cameras, LiDARs, accelerometers, and gyroscopes in mobile devices) in real-time is a big challenge. We follow a holistic, cooperative approach to recognize humans' movements and forecast their trajectories. Heterogeneous open sets of agents, i.e., collaboratively interacting vehicles, infrastructure, and VRUs equipped with mobile devices, exchange information to determine individual models of their surrounding environment, allowing an accurate and reliable forecast of VRU basic movements and trajectories. The collective intelligence of cooperating agents resolves occlusions, implausibilities, and inconsistencies. We developed new methods by considering and combining novel signal processing and modeling techniques with machine learning-based pattern recognition approaches. The cooperation between agents happens on several levels: the VRU perception level, the level of recognized trajectories, or the level of already detected intentions.

  @inbook{zernetsch2024detecting, abstract = {The project Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving (DeCoInt{\$}{\$}^2{\$}{\$}2) focuses on detecting the intentions of vulnerable road users (VRUs) in automated driving using cooperative technologies. Especially in urban areas, VRUs, e.g., pedestrians and cyclists, will continue to play an essential role in mixed traffic. For an accident-free and highly efficient traffic flow with automated vehicles, it is vital to perceive VRUs and their intentions and analyze them similarly to humans when driving and forecasting VRU trajectories. Doing this reliably and robustly with a multimodal sensor system (e.g., cameras, LiDARs, accelerometers, and gyroscopes in mobile devices) in real-time is a big challenge. We follow a holistic, cooperative approach to recognize humans' movements and forecast their trajectories. Heterogeneous open sets of agents, i.e., collaboratively interacting vehicles, infrastructure, and VRUs equipped with mobile devices, exchange information to determine individual models of their surrounding environment, allowing an accurate and reliable forecast of VRU basic movements and trajectories. The collective intelligence of cooperating agents resolves occlusions, implausibilities, and inconsistencies. We developed new methods by considering and combining novel signal processing and modeling techniques with machine learning-based pattern recognition approaches. The cooperation between agents happens on several levels: the VRU perception level, the level of recognized trajectories, or the level of already detected intentions.}, author = {Zernetsch, Stefan and Kress, Viktor and Bieshaar, Maarten and Schneegans, Jan and Reitberger, G{\"u}nther and Fuchs, Erich and Sick, Bernhard and Doll, Konrad}, booktitle = {Cooperatively Interacting Vehicles: Methods and Effects of Automated Cooperation in Traffic}, editor = {Stiller, Christoph and Althoff, Matthias and Burger, Christoph and Deml, Barbara and Eckstein, Lutz and Flemisch, Frank}, keywords = {isac-www}, pages = {35–87}, publisher = {Springer}, title = {Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence as a Basis for Automated Driving}, year = 2024 }
• Spatio-temporal temperature models for growing 2D workpieces during DLD with emissivity compensation for thermal imaging. da Fonseca Pereira, Guilherme; Jelicic, Goran; Kroll, Andreas. In 2024 IEEE Conference on Control Technology and Applications (CCTA), bll 310–315. Newcastle, UK, 2024.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{Pereira-CCTA-2024, address = {Newcastle, UK}, author = {da Fonseca Pereira, Guilherme and Jelicic, Goran and Kroll, Andreas}, booktitle = {2024 IEEE Conference on Control Technology and Applications (CCTA)}, keywords = {isac-www}, month = {August}, pages = {310–315}, title = {Spatio-temporal temperature models for growing 2D workpieces during DLD with emissivity compensation for thermal imaging}, year = 2024 }
• Systematic Evaluation of Uncertainty Calibration in Pretrained Object Detectors. Huseljic, Denis; Herde, Marek; Hahn, Paul; Müjde, Mehmet; Sick, Bernhard. In International Journal of Computer Vision. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  In the field of deep learning based computer vision, the development of deep object detection has led to unique paradigms (e.g., two-stage or set-based) and architectures (e.g., Faster-RCNN or DETR) which enable outstanding performance on challenging benchmark datasets. Despite this, the trained object detectors typically do not reliably assess uncertainty regarding their own knowledge, and the quality of their probabilistic predictions is usually poor. As these are often used to make subsequent decisions, such inaccurate probabilistic predictions must be avoided. In this work, we investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation and conduct detailed analyses assessing the calibration under distributional changes (e.g., distributional shift and application to out-of-distribution data). Furthermore, by investigating the influence of different detector paradigms, post-processing steps, and suitable choices of metrics, we deliver novel insights into why poor detector calibration emerges. Based on these insights, we are able to improve the calibration of a detector by simply finetuning its last layer.

  @article{huseljic2024systematic, abstract = {In the field of deep learning based computer vision, the development of deep object detection has led to unique paradigms (e.g., two-stage or set-based) and architectures (e.g., Faster-RCNN or DETR) which enable outstanding performance on challenging benchmark datasets. Despite this, the trained object detectors typically do not reliably assess uncertainty regarding their own knowledge, and the quality of their probabilistic predictions is usually poor. As these are often used to make subsequent decisions, such inaccurate probabilistic predictions must be avoided. In this work, we investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation and conduct detailed analyses assessing the calibration under distributional changes (e.g., distributional shift and application to out-of-distribution data). Furthermore, by investigating the influence of different detector paradigms, post-processing steps, and suitable choices of metrics, we deliver novel insights into why poor detector calibration emerges. Based on these insights, we are able to improve the calibration of a detector by simply finetuning its last layer.}, author = {Huseljic, Denis and Herde, Marek and Hahn, Paul and Müjde, Mehmet and Sick, Bernhard}, journal = {International Journal of Computer Vision}, keywords = {isac-www}, publisher = {Springer}, title = {Systematic Evaluation of Uncertainty Calibration in Pretrained Object Detectors}, year = 2024 }
• Hierarchical Distributed Scenario-Based Model Predictive Control of Interconnected Microgrids. Schenck, T. Alissa; Hans, Christian Andreas. In 2024 European Control Conference (ECC), bll 2843–2850. 2024.
  • [ BibTeX ]
  @inproceedings{10590752, author = {Schenck, T. Alissa and Hans, Christian Andreas}, booktitle = {2024 European Control Conference (ECC)}, keywords = {isac-www}, pages = {2843-2850}, title = {Hierarchical Distributed Scenario-Based Model Predictive Control of Interconnected Microgrids}, year = 2024 }
• Self-Aware Control for Autonomous Underwater Vehicles. Decke, Jens; Gerland, Florian; Schomberg, Thomas; Wünsch, Olaf; Sick, Bernhard; Gruhl, Christian. In Workshop on self-improving system integration (SISSY), ACSOS, bll 13–14. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  We present a concept to research the implementation of a self-aware control system for underwater cleaning robots. Multiple robots cooperate as a swarm to clean the hull of large container ships while in transit. We utilise physics-informed neural networks to implement an autonomous steering system and combine it with a deep reinforcement learning-based decision and scheduling system. This approach ensures efficient navigation and cleaning operations, adapting to dynamic environmental conditions in real-time. The proposed control system allows the cleaner robots, to enhance fuel efficiency, reduce CO2 emissions, and minimize the environmental impact of maritime transportation.

  @inproceedings{decke2024self, abstract = {We present a concept to research the implementation of a self-aware control system for underwater cleaning robots. Multiple robots cooperate as a swarm to clean the hull of large container ships while in transit. We utilise physics-informed neural networks to implement an autonomous steering system and combine it with a deep reinforcement learning-based decision and scheduling system. This approach ensures efficient navigation and cleaning operations, adapting to dynamic environmental conditions in real-time. The proposed control system allows the cleaner robots, to enhance fuel efficiency, reduce CO2 emissions, and minimize the environmental impact of maritime transportation.}, author = {Decke, Jens and Gerland, Florian and Schomberg, Thomas and Wünsch, Olaf and Sick, Bernhard and Gruhl, Christian}, booktitle = {Workshop on self-improving system integration (SISSY), ACSOS}, keywords = {isac-www}, pages = {13–14}, publisher = {IEEE}, title = {Self-Aware Control for Autonomous Underwater Vehicles}, year = 2024 }
• Fault Detection for Agents in Power Grid Topology Optimization: A Comprehensive Analysis. Lehna, Malte; Hassouna, Mohamed; Degtyar, Dmitry; Tomforde, Sven; Scholz, Christoph. In Machine Learning for Sustainable Power Systems (ML4SPS), ECML. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic chronics and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed chronics of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure detection performance, with an accuracy of 86%. It also correctly identifies in 91% of the time failure and survival observations. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.

  @inproceedings{lehna2024fault, abstract = {Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic chronics and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed chronics of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure detection performance, with an accuracy of 86%. It also correctly identifies in 91% of the time failure and survival observations. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.}, author = {Lehna, Malte and Hassouna, Mohamed and Degtyar, Dmitry and Tomforde, Sven and Scholz, Christoph}, booktitle = {Machine Learning for Sustainable Power Systems (ML4SPS), ECML}, keywords = {isac-www}, note = {(accepted)}, publisher = {Springer}, title = {Fault Detection for Agents in Power Grid Topology Optimization: A Comprehensive Analysis}, year = 2024 }
• LiST: An All-Linear-Layer Spatial-Temporal Feature Extractor with Uncertainty Estimation for RUL Prediction. Huang, Zhixin; Gruhl, Christian; Sick, Bernhard. In IEEE Conference on Industrial Electronics and Applications (ICIEA), bll 1–7. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  In the context of Remaining Useful Life (RUL) prediction for industrial systems, the pursuit of prediction accuracy must be balanced against the hardware costs of model operation and the reliability of prediction results. To resolve these challenges, we introduce LiST, an all-linear-layer spatial-temporal feature extractor integrated with uncertainty estimation, specifically designed for processing sensor multivariate time series (MTS) data. Unlike traditional linear models that flatten MTS and thus neglect their spatial-temporal dependencies, LiST's linear layers act on both the sensor and time dimensions of MTS that can extract spatial and temporal features like GNN and RNN models. Through performance comparisons on four RUL prediction datasets, LiST uses only 66.1% of the parameters, achieves comparable accuracy to state-of-the-art GNN and RNN models, obtains the best results on two datasets with up to a 21.6% improvement in Score, and enhances training efficiency by 3.2 times. Additionally, LiST can predict RUL with uncertainty estimation and precisely disentangle epistemic and aleatoric uncertainties, thus enhancing the model's practicality and reliability in real-world industrial applications.

  @inproceedings{huang2024list, abstract = {In the context of Remaining Useful Life (RUL) prediction for industrial systems, the pursuit of prediction accuracy must be balanced against the hardware costs of model operation and the reliability of prediction results. To resolve these challenges, we introduce LiST, an all-linear-layer spatial-temporal feature extractor integrated with uncertainty estimation, specifically designed for processing sensor multivariate time series (MTS) data. Unlike traditional linear models that flatten MTS and thus neglect their spatial-temporal dependencies, LiST's linear layers act on both the sensor and time dimensions of MTS that can extract spatial and temporal features like GNN and RNN models. Through performance comparisons on four RUL prediction datasets, LiST uses only 66.1% of the parameters, achieves comparable accuracy to state-of-the-art GNN and RNN models, obtains the best results on two datasets with up to a 21.6% improvement in Score, and enhances training efficiency by 3.2 times. Additionally, LiST can predict RUL with uncertainty estimation and precisely disentangle epistemic and aleatoric uncertainties, thus enhancing the model's practicality and reliability in real-world industrial applications.}, author = {Huang, Zhixin and Gruhl, Christian and Sick, Bernhard}, booktitle = {IEEE Conference on Industrial Electronics and Applications (ICIEA)}, keywords = {Uncertainty_Estimation}, pages = {1–7}, publisher = {IEEE}, title = {LiST: An All-Linear-Layer Spatial-Temporal Feature Extractor with Uncertainty Estimation for RUL Prediction}, year = 2024 }
• Time-Series Representation Learning via Heterogeneous Spatial-Temporal Contrasting for Remaining Useful Life Prediction. Huang, Zhixin; He, Yujiang; Nivarthi, Chandana Priya; Gruhl, Christian; Sick, Bernhard. In International Conference on Pattern Recognition (ICPR), bll 1–21. Springer, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Classical contrastive learning paradigms rely on manual augmentations like cropping, masking, dropping, or adding noise randomly to create divergent sample views from original data. However, the choice of which method to manipulate samples is often subjective and may destroy the latent pattern of the sample. In response, this paper introduces a novel contrastive learning paradigm without choosing sample view augmentation methods, termed Heterogeneous Spatial-Temporal Representation Contrasting (HSTRC). Instead of sample view augmentation, we employ dual branches with a heterogeneous spatial-temporal flipped structure to extract two distinct hidden feature views from the same source data, which avoids disturbing the original time series. Leveraging a combination of cross-branch spatial-temporal contrastive and projected feature contrastive loss functions, HSTRC can effectively extract robust representations from unlabeled time series data. Remarkably, by only fine-tuning the fully connected layers on top of extracted representations by HSTRC, we achieve the best performance across several Remaining Useful Life prediction datasets, showing up to 19.2% improvements over the state-of-the-art supervised learning methods and classical contrastive learning paradigms. Besides, further intensive experiments demonstrate HSTRC's effectiveness in active learning, out-of-distribution testing, and transfer learning scenarios.

  @inproceedings{huang2024time, abstract = {Classical contrastive learning paradigms rely on manual augmentations like cropping, masking, dropping, or adding noise randomly to create divergent sample views from original data. However, the choice of which method to manipulate samples is often subjective and may destroy the latent pattern of the sample. In response, this paper introduces a novel contrastive learning paradigm without choosing sample view augmentation methods, termed Heterogeneous Spatial-Temporal Representation Contrasting (HSTRC). Instead of sample view augmentation, we employ dual branches with a heterogeneous spatial-temporal flipped structure to extract two distinct hidden feature views from the same source data, which avoids disturbing the original time series. Leveraging a combination of cross-branch spatial-temporal contrastive and projected feature contrastive loss functions, HSTRC can effectively extract robust representations from unlabeled time series data. Remarkably, by only fine-tuning the fully connected layers on top of extracted representations by HSTRC, we achieve the best performance across several Remaining Useful Life prediction datasets, showing up to 19.2% improvements over the state-of-the-art supervised learning methods and classical contrastive learning paradigms. Besides, further intensive experiments demonstrate HSTRC's effectiveness in active learning, out-of-distribution testing, and transfer learning scenarios.}, author = {Huang, Zhixin and He, Yujiang and Nivarthi, Chandana Priya and Gruhl, Christian and Sick, Bernhard}, booktitle = {International Conference on Pattern Recognition (ICPR)}, keywords = {Temporal-Spatial_Contrasting}, pages = {1–21}, publisher = {Springer}, title = {Time-Series Representation Learning via Heterogeneous Spatial-Temporal Contrasting for Remaining Useful Life Prediction}, year = 2024 }
• Steering Towards Maritime Safety with True Motion Predictions Ensemble. Al-Falouji, Ghassan; Beyer, Tom; Gao, Shang; Tomforde, Sven. In Workshop on self-improving system integration (SISSY), ACSOS, bll 7–12. IEEE, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  Maritime transportation is vital for global trade, yet safety challenges persist, especially with increasing vessel density and narrow waterways. The standard Closest Point of Approach (CPA) method, used to estimate collision risk, has limitations due to its reliance on constant Course Over Ground (COG) and Speed Over Ground (SOG) assumptions, neglecting dynamic vessel behaviours, true ship dimensions, and environmental influences. Addressing these limitations, this paper introduces an enhanced CPA (eCPA) method that counters these deficits by incorporating ship trajectory predictions, probabilistic risk assessment, and real-time data integration. The eCPA method includes user- adjustable tools for trajectory interpolation, prediction, and anomaly elimination, making it adaptable to various maritime scenarios. Extensive testing in encounter scenarios demonstrates the effectiveness of our approach in improving maritime safety through robust and probabilistic estimation of encounter points. This method aligns with the Self-Integration and Self-Organising Systems (SISSY) initiative, exhibiting how autonomous maritime operations can benefit from self-improving and self-integrating systems to enhance navigation safety and efficiency.

  @inproceedings{alfalouji2024steering, abstract = {Maritime transportation is vital for global trade, yet safety challenges persist, especially with increasing vessel density and narrow waterways. The standard Closest Point of Approach (CPA) method, used to estimate collision risk, has limitations due to its reliance on constant Course Over Ground (COG) and Speed Over Ground (SOG) assumptions, neglecting dynamic vessel behaviours, true ship dimensions, and environmental influences. Addressing these limitations, this paper introduces an enhanced CPA (eCPA) method that counters these deficits by incorporating ship trajectory predictions, probabilistic risk assessment, and real-time data integration. The eCPA method includes user- adjustable tools for trajectory interpolation, prediction, and anomaly elimination, making it adaptable to various maritime scenarios. Extensive testing in encounter scenarios demonstrates the effectiveness of our approach in improving maritime safety through robust and probabilistic estimation of encounter points. This method aligns with the Self-Integration and Self-Organising Systems (SISSY) initiative, exhibiting how autonomous maritime operations can benefit from self-improving and self-integrating systems to enhance navigation safety and efficiency.}, author = {Al-Falouji, Ghassan and Beyer, Tom and Gao, Shang and Tomforde, Sven}, booktitle = {Workshop on self-improving system integration (SISSY), ACSOS}, keywords = {Systems}, pages = {7–12}, publisher = {IEEE}, title = {Steering Towards Maritime Safety with True Motion Predictions Ensemble}, year = 2024 }
• Characterization of the angular distribution of the radiance and homogeneity of a integrating sphere and a parabolic reflector in the middle- and long-wavelength infrared with a self-developed gonioradiometer. Noori, Hoschang; Schmoll, Robert; Kroll, Andreas. In SPIE Optical Engineering + Applications (OP24O). SPIE, San Diego, 2024.
  • [ BibTeX ]
  @inproceedings{Noori2024a, address = {San Diego}, author = {Noori, Hoschang and Schmoll, Robert and Kroll, Andreas}, booktitle = {SPIE Optical Engineering + Applications (OP24O)}, keywords = {isac-www}, month = {September}, organization = {SPIE}, title = {Characterization of the angular distribution of the radiance and homogeneity of a integrating sphere and a parabolic reflector in the middle- and long-wavelength infrared with a self-developed gonioradiometer}, year = 2024 }
• An airborne measurement system to detect, locate and quantify methane emissions with attitude-based wind speed estimation. Kistner, Lars; Schmoll, Robert; Kroll, Andreas. In Journal of Sensors and Sensor Systems, 13(2), bll 211–218. 2024.
  • [ BibTeX ]
  • [ URL ]
  @article{Kistner2024, author = {Kistner, Lars and Schmoll, Robert and Kroll, Andreas}, journal = {Journal of Sensors and Sensor Systems}, keywords = {isac-www}, number = 2, pages = {211–218}, title = {An airborne measurement system to detect, locate and quantify methane emissions with attitude-based wind speed estimation}, volume = 13, year = 2024 }
• A Digital Twin for Part Quality Prediction and Control in Plastic Injection Molding. Rehmer, Alexander; Klute, Marco; Heim, Hans-Peter; Kroll, Andreas. In Modeling, Identification and Control for Cyber-Physical Systems Towards Industry 4.0, 1st ed, P. Mercorelli, W. Zhang, H. Nemati, Y. Zhang (reds.), bll 79–109. Academic Press, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The plastic injection molding process has been established as the most widespread manufacturing process in the plastic processing industry. It is employed by almost 70 % of all plastic processing companies. Among the decisive factors contributing to its prevalence are the ability to manufacture parts with intricate geometries and a high degree of automation. There are approaches of varying complexity to control the part quality to reduce waste and increase the efficiency of the process: The industry standard is the control of the so-called machine-variables i.e. the process variables that are measured on the machine side of the process. This does not take into account any variables that reflect the true state of the emerging part. For this reason the scientific community aims to control process variables that are measured cavity-side, more precisely the pressure in the mold cavity. However the implementation of pressure control requires significant control knowledge and is not suitable for large-scale industrial application. The objective of this contribution is therefore to transform an ordinary machine-variable controlled injection molding machine to a Cyber Physical Production System (CPPS) via augmentation by a digital twin (DT). The DT will predict the part quality from process variables. To this end a state-of-the-art industrial injection molding machine was equipped with additional sensors that measure in-cavity process variables. Moreover an in-line quality measuring cell was added. By doing so all machine, processes and quality data required for data-driven modeling and perspectively control are acquired. Subsequently an internal dynamics approach for predicting final batch quality from process value trajectories is proposed and compared to the current state-of-the-art modeling approaches in a case study.

  @incollection{EmergMeth_Rehmer_2022, abstract = {The plastic injection molding process has been established as the most widespread manufacturing process in the plastic processing industry. It is employed by almost 70 \% of all plastic processing companies. Among the decisive factors contributing to its prevalence are the ability to manufacture parts with intricate geometries and a high degree of automation. There are approaches of varying complexity to control the part quality to reduce waste and increase the efficiency of the process: The industry standard is the control of the so-called machine-variables i.e. the process variables that are measured on the machine side of the process. This does not take into account any variables that reflect the true state of the emerging part. For this reason the scientific community aims to control process variables that are measured cavity-side, more precisely the pressure in the mold cavity. However the implementation of pressure control requires significant control knowledge and is not suitable for large-scale industrial application. The objective of this contribution is therefore to transform an ordinary machine-variable controlled injection molding machine to a Cyber Physical Production System (CPPS) via augmentation by a digital twin (DT). The DT will predict the part quality from process variables. To this end a state-of-the-art industrial injection molding machine was equipped with additional sensors that measure in-cavity process variables. Moreover an in-line quality measuring cell was added. By doing so all machine, processes and quality data required for data-driven modeling and perspectively control are acquired. Subsequently an internal dynamics approach for predicting final batch quality from process value trajectories is proposed and compared to the current state-of-the-art modeling approaches in a case study.}, author = {Rehmer, Alexander and Klute, Marco and Heim, Hans-Peter and Kroll, Andreas}, booktitle = {Modeling, Identification and Control for Cyber-Physical Systems Towards Industry 4.0}, chapter = 4, edition = 1, editor = {Mercorelli, Paolo and Zhang, Weicun and Nemati, Hamidreza and Zhang, YuMing}, keywords = {quality}, pages = {79–109}, publisher = {Academic Press}, series = {Emerging Methodologies and Applications in Modelling, Identification and Control}, title = {A Digital Twin for Part Quality Prediction and Control in Plastic Injection Molding}, year = 2024 }
• Datengetriebene Modellierung und Merkmalsselektion für die automatische Kalibrierung eines Sensorsystems für die mikromagnetische Materialcharakterisierung. Felix Wittich, Andreas Kroll. In Proceedings 34. Workshop Computational Intelligence: Berlin 21.-22. November 2024, Vol. 21, bll 21–26. KIT Scientific Publishing, 2024.
  • [ BibTeX ]
  @inproceedings{Wittich2024GMA, author = {Felix Wittich, Andreas Kroll}, booktitle = {Proceedings 34. Workshop Computational Intelligence: Berlin 21.-22. November 2024}, keywords = {isac-www}, organization = {KIT Scientific Publishing}, pages = {21–26}, title = {Datengetriebene Modellierung und Merkmalsselektion für die automatische Kalibrierung eines Sensorsystems für die mikromagnetische Materialcharakterisierung}, volume = 21, year = 2024 }
• PrOuD: Probabilistic Outlier Detection Solution for Time Series Analysis on Real-world Photovoltaic Inverters. He, Yujiang; Huang, Zhixin; Vogt, Stephan; Sick, Bernhard. In Energies (MDPI), 17(1), bl 64. MDPI, 2024.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Anomaly detection methods applied to time series are mostly viewed as a black box, which solely provides a deterministic answer to the detected target. Without a convincing explanation, domain experts can hardly trust the detection results and conduct a further time series diagnosis in real-world applications. To overcome the challenge, we mathematically analyzed the sources of anomalies and novelties in multivariate time series as well as their relationships from the perspective of Gaussian-distributed non-stationary noise. Furthermore, we proposed mathematical methods to generate artificial time series and synthetic anomalies, with the goal of solving the problem that it is difficult to train and evaluate models in real-world applications due to the lack of sufficient data. In addition, we designed extbfProbabilistic extbfOutlier extbfDetection (PrOuD), which is a general solution to provide interpretable detection results to assist domain experts in time series analysis. PrOuD can convert a predictive uncertainty of a trained model about a time series value into an estimated uncertainty of the detected outlier through Monte Carlo Estimation. The experimental results obtained on both artificial time series and real-world photovoltaic inverter data demonstrated that the proposed solution could detect emerging anomalies accurately and quickly. The implemented PrOuD demo case shows the potential to makes the detection results of the existing detection methods more convincing so that domain experts can more efficiently complete their tasks, such as time series diagnosis and anomalous pattern clustering.

  @article{he2023proud, abstract = {Anomaly detection methods applied to time series are mostly viewed as a black box, which solely provides a deterministic answer to the detected target. Without a convincing explanation, domain experts can hardly trust the detection results and conduct a further time series diagnosis in real-world applications. To overcome the challenge, we mathematically analyzed the sources of anomalies and novelties in multivariate time series as well as their relationships from the perspective of Gaussian-distributed non-stationary noise. Furthermore, we proposed mathematical methods to generate artificial time series and synthetic anomalies, with the goal of solving the problem that it is difficult to train and evaluate models in real-world applications due to the lack of sufficient data. In addition, we designed extbf{Pr}obabilistic extbf{Ou}tlier extbf{D}etection (PrOuD), which is a general solution to provide interpretable detection results to assist domain experts in time series analysis. PrOuD can convert a predictive uncertainty of a trained model about a time series value into an estimated uncertainty of the detected outlier through Monte Carlo Estimation. The experimental results obtained on both artificial time series and real-world photovoltaic inverter data demonstrated that the proposed solution could detect emerging anomalies accurately and quickly. The implemented PrOuD demo case shows the potential to makes the detection results of the existing detection methods more convincing so that domain experts can more efficiently complete their tasks, such as time series diagnosis and anomalous pattern clustering.}, author = {He, Yujiang and Huang, Zhixin and Vogt, Stephan and Sick, Bernhard}, journal = {Energies (MDPI)}, keywords = {time_series}, number = 1, pages = 64, publisher = {MDPI}, title = {PrOuD: Probabilistic Outlier Detection Solution for Time Series Analysis on Real-world Photovoltaic Inverters}, volume = 17, year = 2024 }

2023

• A Predictive Operation Controller for an Electro-Thermal Microgrid Utilizing Variable Flow Temperatures. Rose, Max; Hans, Christian A.; Schiffer, Johannes. In IFAC-PapersOnLine, 56(2), bll 5444–5450. Elsevier BV, 2023.
  • [ BibTeX ]
  • [ URL ]
  @article{Rose_2023, author = {Rose, Max and Hans, Christian A. and Schiffer, Johannes}, journal = {IFAC-PapersOnLine}, keywords = {isac-www}, number = 2, pages = {5444–5450}, publisher = {Elsevier BV}, title = {A Predictive Operation Controller for an Electro-Thermal Microgrid Utilizing Variable Flow Temperatures}, volume = 56, year = 2023 }
• Multi-annotator Deep Learning: A Probabilistic Framework for Classification. Herde, Marek; Huseljic, Denis; Sick, Bernhard. In Transactions on Machine Learning Research. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Solving complex classification tasks using deep neural networks typically requires large amounts of annotated data. However, corresponding class labels are noisy when provided by error-prone annotators, e.g., crowdworkers. Training standard deep neural networks leads to subpar performances in such multi-annotator supervised learning settings. We address this issue by presenting a probabilistic training framework named multi-annotator deep learning (MaDL). A downstream ground truth and an annotator performance model are jointly trained in an end-to-end learning approach. The ground truth model learns to predict instances' true class labels, while the annotator performance model infers probabilistic estimates of annotators' performances. A modular network architecture enables us to make varying assumptions regarding annotators' performances, e.g., an optional class or instance dependency. Further, we learn annotator embeddings to estimate annotators' densities within a latent space as proxies of their potentially correlated annotations. Together with a weighted loss function, we improve the learning from correlated annotation patterns. In a comprehensive evaluation, we examine three research questions about multi-annotator supervised learning. Our findings show MaDL's state-of-the-art performance and robustness against many correlated, spamming annotators.

  @article{herde2023multi, abstract = {Solving complex classification tasks using deep neural networks typically requires large amounts of annotated data. However, corresponding class labels are noisy when provided by error-prone annotators, e.g., crowdworkers. Training standard deep neural networks leads to subpar performances in such multi-annotator supervised learning settings. We address this issue by presenting a probabilistic training framework named multi-annotator deep learning (MaDL). A downstream ground truth and an annotator performance model are jointly trained in an end-to-end learning approach. The ground truth model learns to predict instances' true class labels, while the annotator performance model infers probabilistic estimates of annotators' performances. A modular network architecture enables us to make varying assumptions regarding annotators' performances, e.g., an optional class or instance dependency. Further, we learn annotator embeddings to estimate annotators' densities within a latent space as proxies of their potentially correlated annotations. Together with a weighted loss function, we improve the learning from correlated annotation patterns. In a comprehensive evaluation, we examine three research questions about multi-annotator supervised learning. Our findings show MaDL's state-of-the-art performance and robustness against many correlated, spamming annotators.}, author = {Herde, Marek and Huseljic, Denis and Sick, Bernhard}, journal = {Transactions on Machine Learning Research}, keywords = {Crowdsourcing}, title = {Multi-annotator Deep Learning: A Probabilistic Framework for Classification}, year = 2023 }
• A Unified Approach to Communication Delay and Communication Frequency in Distributed State Estimation of Linear Systems. Mast, J.; Liu, Z.; Wang, Z.; Stursberg, O. In IEEE Control Systems Letters, Volume 7, bll 2755–2760. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This letter introduces a novel method to address communication delay and communication frequency in distributed state estimation using a unified structure. Starting from a general type of delay, which can be time-varying, unknown, and different for each channel, two estimation strategies are proposed which use either a unified upper-bound of the delay in all channels, or a sender specific upper-bound in order to improve the performance. The strategies guarantee convergence of the estimation error, and they can be applied to study how communication frequency (or asynchronous communication) affects the convergence rate of the estimation. Thus, the task of determining a necessary communication frequency for obtaining specified performance of distributed estimation is considered, and this task is approached as a special case of the one tailored to delay. Effectiveness of the techniques is confirmed by numerical examples for both types of tasks.

  @article{mast2023unified, abstract = {This letter introduces a novel method to address communication delay and communication frequency in distributed state estimation using a unified structure. Starting from a general type of delay, which can be time-varying, unknown, and different for each channel, two estimation strategies are proposed which use either a unified upper-bound of the delay in all channels, or a sender specific upper-bound in order to improve the performance. The strategies guarantee convergence of the estimation error, and they can be applied to study how communication frequency (or asynchronous communication) affects the convergence rate of the estimation. Thus, the task of determining a necessary communication frequency for obtaining specified performance of distributed estimation is considered, and this task is approached as a special case of the one tailored to delay. Effectiveness of the techniques is confirmed by numerical examples for both types of tasks.}, author = {Mast, J. and Liu, Z. and Wang, Z. and Stursberg, O.}, journal = {IEEE Control Systems Letters}, keywords = {isac-www}, pages = {2755-2760}, title = {A Unified Approach to Communication Delay and Communication Frequency in Distributed State Estimation of Linear Systems}, volume = {Volume 7}, year = 2023 }
• Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents. Lehna, Malte; Viebahn, Jan; Marot, Antoine; Tomforde, Sven; Scholz, Christoph. In Energy and AI, 14, bl 100276. Energy and AI, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network (L2RPN) challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of both the rule-based and the RL agent become more diversified.

  @article{lehna2023managing, abstract = {The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network (L2RPN) challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of both the rule-based and the RL agent become more diversified.}, author = {Lehna, Malte and Viebahn, Jan and Marot, Antoine and Tomforde, Sven and Scholz, Christoph}, journal = {Energy and AI}, keywords = {ProximalPolicyOptimisation}, pages = 100276, publisher = {Energy and AI}, title = {Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents}, volume = 14, year = 2023 }
• Sensor Equivariance by LiDAR Projection Images. Reichert, Hannes; Hetzel, Manuel; Schreck, Steven; Doll, Konrad; Sick, Bernhard. In IEEE Intelligent Vehicles Symposium (IV), bll 1–6. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we propose an architecture for processing this data in an instance segmentation framework. We focus specifically on LiDAR as a key sensor modality for machine vision tasks and highly automated driving (HAD). Through an experimental setup in a controlled synthetic environment, we identify a bias on sensor resolution and field of view and demonstrate that our proposed method can reduce said bias for the task of LiDAR instance segmentation. Furthermore, we define our method such that it can be applied to other projection-based sensors, such as cameras. To promote transparency, we make our code and dataset publicly available. This method shows the potential to improve performance and robustness in various machine vision tasks that utilize projection-based sensors.

  @inproceedings{reichert2023sensor, abstract = {In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we propose an architecture for processing this data in an instance segmentation framework. We focus specifically on LiDAR as a key sensor modality for machine vision tasks and highly automated driving (HAD). Through an experimental setup in a controlled synthetic environment, we identify a bias on sensor resolution and field of view and demonstrate that our proposed method can reduce said bias for the task of LiDAR instance segmentation. Furthermore, we define our method such that it can be applied to other projection-based sensors, such as cameras. To promote transparency, we make our code and dataset publicly available. This method shows the potential to improve performance and robustness in various machine vision tasks that utilize projection-based sensors.}, author = {Reichert, Hannes and Hetzel, Manuel and Schreck, Steven and Doll, Konrad and Sick, Bernhard}, booktitle = {IEEE Intelligent Vehicles Symposium (IV)}, keywords = {isac-www}, pages = {1–6}, publisher = {IEEE}, title = {Sensor Equivariance by {LiDAR} Projection Images}, year = 2023 }
• The IMPTC Dataset: An Infrastructural Multi-Person Trajectory and Context Dataset. Hetzel, Manuel; Reichert, Hannes; Reitberger, Günther; Doll, Konrad; Sick, Bernhard; Fuchs, Erich. In IEEE Intelligent Vehicles Symposium (IV), bll 1–7. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Inner-city intersections are among the most critical traffic areas for injury and fatal accidents. Automated vehicles struggle with the complex and hectic everyday life within those areas. Sensor-equipped smart infrastructures, which can cooperate with vehicles, can benefit automated traffic by extending the perception capabilities of drivers and vehicle perception systems. Additionally, they offer the opportunity to gather reproducible and precise data of a holistic scene understanding, including context information as a basis for training algorithms for various applications in automated traffic. Therefore, we introduce the Infrastructural Multi-Person Trajectory and Context Dataset (IMPTC). We use an intelligent public inner-city intersection in Germany with visual sensor technology. A multi-view camera and LiDAR system perceives traffic situations and road users’ behavior. Additional sensors monitor contextual information like weather, lighting, and traffic light signal status. The data acquisition system focuses on Vulnerable Road Users (VRUs) and multiagent interaction. The resulting dataset consists of eight hours of measurement data. It contains over 2,500 VRU trajectories, including pedestrians, cyclists, e-scooter riders, strollers, and wheelchair users, and over 20,000 vehicle trajectories at different day times, weather conditions, and seasons. In addition, to enable the entire stack of research capabilities, the dataset includes all data, starting from the sensor-, calibration- and detection data until trajectory and context data. The dataset is continuously expanded and is available online for non-commercial research at https://github.com/kav-institute/imptc-dataset

  @inproceedings{hetzel2023imptc, abstract = {Inner-city intersections are among the most critical traffic areas for injury and fatal accidents. Automated vehicles struggle with the complex and hectic everyday life within those areas. Sensor-equipped smart infrastructures, which can cooperate with vehicles, can benefit automated traffic by extending the perception capabilities of drivers and vehicle perception systems. Additionally, they offer the opportunity to gather reproducible and precise data of a holistic scene understanding, including context information as a basis for training algorithms for various applications in automated traffic. Therefore, we introduce the Infrastructural Multi-Person Trajectory and Context Dataset (IMPTC). We use an intelligent public inner-city intersection in Germany with visual sensor technology. A multi-view camera and LiDAR system perceives traffic situations and road users’ behavior. Additional sensors monitor contextual information like weather, lighting, and traffic light signal status. The data acquisition system focuses on Vulnerable Road Users (VRUs) and multiagent interaction. The resulting dataset consists of eight hours of measurement data. It contains over 2,500 VRU trajectories, including pedestrians, cyclists, e-scooter riders, strollers, and wheelchair users, and over 20,000 vehicle trajectories at different day times, weather conditions, and seasons. In addition, to enable the entire stack of research capabilities, the dataset includes all data, starting from the sensor-, calibration- and detection data until trajectory and context data. The dataset is continuously expanded and is available online for non-commercial research at https://github.com/kav-institute/imptc-dataset}, author = {Hetzel, Manuel and Reichert, Hannes and Reitberger, Günther and Doll, Konrad and Sick, Bernhard and Fuchs, Erich}, booktitle = {IEEE Intelligent Vehicles Symposium (IV)}, keywords = {isac-www}, pages = {1–7}, publisher = {IEEE}, title = {The {IMPTC} Dataset: An Infrastructural Multi-Person Trajectory and Context Dataset}, year = 2023 }
• Unraveling the Complexity of Splitting Sequential Data: Tackling Challenges in Video and Time Series Analysis. Botache, Diego; Dingel, Kristina; Huhnstock, Rico; Ehresmann, Arno; Sick, Bernhard. In arXiv e-prints, bl arXiv:2307.14294. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Splitting of sequential data, such as videos and time series, is an essential step in various data analysis tasks, including object tracking and anomaly detection. However, splitting sequential data presents a variety of challenges that can impact the accuracy and reliability of subsequent analyses. This concept article examines the challenges associated with splitting sequential data, including data acquisition, data representation, split ratio selection, setting up quality criteria, and choosing suitable selection strategies. We explore these challenges through two real-world examples: motor test benches and particle tracking in liquids.

  @article{botache2023unraveling, abstract = {Splitting of sequential data, such as videos and time series, is an essential step in various data analysis tasks, including object tracking and anomaly detection. However, splitting sequential data presents a variety of challenges that can impact the accuracy and reliability of subsequent analyses. This concept article examines the challenges associated with splitting sequential data, including data acquisition, data representation, split ratio selection, setting up quality criteria, and choosing suitable selection strategies. We explore these challenges through two real-world examples: motor test benches and particle tracking in liquids.}, author = {Botache, Diego and Dingel, Kristina and Huhnstock, Rico and Ehresmann, Arno and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {Sensor-Data}, pages = {arXiv:2307.14294}, title = {Unraveling the Complexity of Splitting Sequential Data: Tackling Challenges in Video and Time Series Analysis}, year = 2023 }
• Domain Imaging in Periodic Submicron Wide Nanostructures by Digital Drift Correction in Kerr Microscopy. Akhundzada, Sapida; Dingel, Kristina; Bischof, David; Janzen, Christian; Sick, Bernhard; Ehresmann, Arno. In Advanced Photonics Research, 4(10), bl 2300170. Wiley, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Magneto-optical Kerr microscopy is a powerful method for imaging magnetic domains. Even though domain imaging below the diffractive resolution limit is possible, such investigations are getting increasingly complex with decreasing structure size due to the decreasing Kerr contrast. As magnetic domain images free of topographical artifacts are obtained by subtracting a reference image from the actual image, the corresponding challenges are additionally increased by unavoidable sample motion in the time interval between acquiring the two images. Software-based drift corrections typically rely on a unique structure in the image's region of interest (ROI), recognized automatically or selected manually by the user. By digital image shifting, the ROI positions in the actual and reference images are aligned, and the sample motion is compensated. For magnetic domain imaging in periodically arranged micro- or nano-objects, unique topographical features are not given, making the drift correction by ROIs difficult, often even impossible. Herein, a novel software-based approach is presented for drift corrections to image domains with features close/below the optical resolution limit and for investigating periodically arranged micro- or nano-objects without utilizing ROIs. High-contrast images are obtained, enabling the characterization of periodically arranged 1D, 2D, and 3D magnetic objects with lateral dimensions below 100 nm.

  @article{akhundzada2023domain, abstract = {Magneto-optical Kerr microscopy is a powerful method for imaging magnetic domains. Even though domain imaging below the diffractive resolution limit is possible, such investigations are getting increasingly complex with decreasing structure size due to the decreasing Kerr contrast. As magnetic domain images free of topographical artifacts are obtained by subtracting a reference image from the actual image, the corresponding challenges are additionally increased by unavoidable sample motion in the time interval between acquiring the two images. Software-based drift corrections typically rely on a unique structure in the image's region of interest (ROI), recognized automatically or selected manually by the user. By digital image shifting, the ROI positions in the actual and reference images are aligned, and the sample motion is compensated. For magnetic domain imaging in periodically arranged micro- or nano-objects, unique topographical features are not given, making the drift correction by ROIs difficult, often even impossible. Herein, a novel software-based approach is presented for drift corrections to image domains with features close/below the optical resolution limit and for investigating periodically arranged micro- or nano-objects without utilizing ROIs. High-contrast images are obtained, enabling the characterization of periodically arranged 1D, 2D, and 3D magnetic objects with lateral dimensions below 100 nm.}, author = {Akhundzada, Sapida and Dingel, Kristina and Bischof, David and Janzen, Christian and Sick, Bernhard and Ehresmann, Arno}, journal = {Advanced Photonics Research}, keywords = {isac-www}, number = 10, pages = 2300170, publisher = {Wiley}, title = {Domain Imaging in Periodic Submicron Wide Nanostructures by Digital Drift Correction in Kerr Microscopy}, volume = 4, year = 2023 }
• Who knows best? A Case Study on Intelligent Crowdworker Selection via Deep Learning. Herde, Marek; Huseljic, Denis; Sick, Bernhard; Bretschneider, Ulrich; Oeste-Reiß, Sarah. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 14–18. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Crowdworking is a popular approach for annotating large amounts of data to train deep neural networks. However, parts of the annotations are often erroneous. In a case study, we demonstrate how an intelligent crowdworker selection via deep learning reduces the number of erroneous annotations and, thus, the annotation costs of obtaining reliable data for training deep neural networks.

  @inproceedings{herde2023who, abstract = {Crowdworking is a popular approach for annotating large amounts of data to train deep neural networks. However, parts of the annotations are often erroneous. In a case study, we demonstrate how an intelligent crowdworker selection via deep learning reduces the number of erroneous annotations and, thus, the annotation costs of obtaining reliable data for training deep neural networks.}, author = {Herde, Marek and Huseljic, Denis and Sick, Bernhard and Bretschneider, Ulrich and Oeste-Reiß, Sarah}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {ActiveLearning}, pages = {14–18}, title = {Who knows best? A Case Study on Intelligent Crowdworker Selection via Deep Learning}, year = 2023 }
• On the Design of Limit Cycles of Planar Switching Affine Systems. Hanke, N.; Stursberg, O. In Proc. of the 2023 European Control Conference, bll 2251–2256. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In the context of studying periodic processes, this paper investigates first under which conditions switching affine systems in the plane generate stable limit cycles. Based on these conditions, a design methodology is proposed by which the phase portraits of the switching systems are determined to obtain globally stable limit cycles from simple specifications, such as given amplitudes and frequencies of desired oscillations. As an application, the paper finally shows that an oscillator model can be derived with a small effort from data measured for an unknown oscillating system.

  @article{nils2023design, abstract = {In the context of studying periodic processes, this paper investigates first under which conditions switching affine systems in the plane generate stable limit cycles. Based on these conditions, a design methodology is proposed by which the phase portraits of the switching systems are determined to obtain globally stable limit cycles from simple specifications, such as given amplitudes and frequencies of desired oscillations. As an application, the paper finally shows that an oscillator model can be derived with a small effort from data measured for an unknown oscillating system.}, author = {Hanke, N. and Stursberg, O.}, journal = {Proc. of the 2023 European Control Conference}, keywords = {isac-www}, pages = {2251-2256}, title = {On the Design of Limit Cycles of Planar Switching Affine Systems.}, year = 2023 }
• Exploring the Potential of Optimal Active Learning via a Non-myopic Oracle Policy. Sandrock, Christoph; Herde, Marek; Kottke, Daniel; Sick, Bernhard. In Discovery Science (DS), bll 265–276. Springer, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Active learning aims to reduce the amount of labeled data while maximizing machine learning models’ performances. Currently, there is sparse research on the potential of an optimal active learning strategy. Therefore, we propose a non-myopic oracle policy that accesses the true labels of the data pool to approximate an optimal active learning strategy. We evaluate how the hyperparameters of this oracle policy influence its performance and empirically demonstrate that it is an upper baseline for common active learning strategies while being faster than a state-of-the-art oracle policy. For the sake of reproducibility, all the code related to our research is publicly available on our GitHub repository at https://github.com/ies-research/non-myopic-oracle-policy.

  @inproceedings{sandrock2023exploring, abstract = {Active learning aims to reduce the amount of labeled data while maximizing machine learning models’ performances. Currently, there is sparse research on the potential of an optimal active learning strategy. Therefore, we propose a non-myopic oracle policy that accesses the true labels of the data pool to approximate an optimal active learning strategy. We evaluate how the hyperparameters of this oracle policy influence its performance and empirically demonstrate that it is an upper baseline for common active learning strategies while being faster than a state-of-the-art oracle policy. For the sake of reproducibility, all the code related to our research is publicly available on our GitHub repository at https://github.com/ies-research/non-myopic-oracle-policy.}, author = {Sandrock, Christoph and Herde, Marek and Kottke, Daniel and Sick, Bernhard}, booktitle = {Discovery Science (DS)}, keywords = {Classification}, pages = {265–276}, publisher = {Springer}, title = {Exploring the Potential of Optimal Active Learning via a Non-myopic Oracle Policy}, year = 2023 }
• Role of Hyperparameters in Deep Active Learning. Huseljic, Denis; Herde, Marek; Hahn, Paul; Sick, Bernhard. In Workshop on Interactive Adaptive Learning (IAL), ECML PKDD, bll 19–24. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Despite considerable research on pool-based deep active learning (DAL), deploying DAL into real applications still has several challenges. A frequently neglected aspect is the choice of training hyperparameters (HPs), such as the learning rate. Since these HPs determine how the deep neural network learns in each cycle, they must be chosen carefully. In this article, we analyze the role of HPs in DAL. We find that optimizing HPs reduces the performance gap between DAL strategies. Conversely, we highlight challenges in finding optimal HPs when using datasets selected via DAL strategies.

  @inproceedings{huseljic2023role, abstract = {Despite considerable research on pool-based deep active learning (DAL), deploying DAL into real applications still has several challenges. A frequently neglected aspect is the choice of training hyperparameters (HPs), such as the learning rate. Since these HPs determine how the deep neural network learns in each cycle, they must be chosen carefully. In this article, we analyze the role of HPs in DAL. We find that optimizing HPs reduces the performance gap between DAL strategies. Conversely, we highlight challenges in finding optimal HPs when using datasets selected via DAL strategies.}, author = {Huseljic, Denis and Herde, Marek and Hahn, Paul and Sick, Bernhard}, booktitle = {Workshop on Interactive Adaptive Learning (IAL), ECML PKDD}, keywords = {isac-www}, pages = {19–24}, title = {Role of Hyperparameters in Deep Active Learning}, year = 2023 }
• On the Reliability of Automated Analysis of Fracture Surfaces Using a Novel Computer Vision-Based Tool. Engelhardt, Anna; Decke, Jens; Meier, David; Dulig, Franz; Ragunathan, Rishan; Wegener, Thomas; Sick, Bernhard; Niendorf, Thomas. In Advanced Engineering Materials, 25(21), bl 2300876. Wiley, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Fracture surface analysis is of utmost importance with respect to structural integrity of metallic materials. This especially holds true for additively manufactured materials. Despite an increasing trend of automatization of testing methods, the analysis and classification of fatigue fracture surface images is commonly done manually by experts. Although this leads to correct results in most cases, it has several disadvantages, e.g., the need of a huge knowledge base to interpret images correctly. In present work, an unsupervised tool for analysis of overview images of fatigue fracture surface images is developed to support nonexperienced users to identify the origin of the fracture. The tool is developed using fracture surface images of additively manufactured Ti6Al4V specimens fatigued in the high-cycle-fatigue regime and is based on the identification of river marks. Several recording parameters seem to have no significant influence on the results as long as preprocessing settings are adapted. Moreover, it is possible to analyze images of other materials with the tool as long as the fracture surfaces contain river marks. However, special features like multiple origins or origins located in direct vicinity to the surface, e.g., caused by increased plastic strains, require a further tool development or alternative approaches.

  @article{engelhardt2023reliability, abstract = {Fracture surface analysis is of utmost importance with respect to structural integrity of metallic materials. This especially holds true for additively manufactured materials. Despite an increasing trend of automatization of testing methods, the analysis and classification of fatigue fracture surface images is commonly done manually by experts. Although this leads to correct results in most cases, it has several disadvantages, e.g., the need of a huge knowledge base to interpret images correctly. In present work, an unsupervised tool for analysis of overview images of fatigue fracture surface images is developed to support nonexperienced users to identify the origin of the fracture. The tool is developed using fracture surface images of additively manufactured Ti6Al4V specimens fatigued in the high-cycle-fatigue regime and is based on the identification of river marks. Several recording parameters seem to have no significant influence on the results as long as preprocessing settings are adapted. Moreover, it is possible to analyze images of other materials with the tool as long as the fracture surfaces contain river marks. However, special features like multiple origins or origins located in direct vicinity to the surface, e.g., caused by increased plastic strains, require a further tool development or alternative approaches.}, author = {Engelhardt, Anna and Decke, Jens and Meier, David and Dulig, Franz and Ragunathan, Rishan and Wegener, Thomas and Sick, Bernhard and Niendorf, Thomas}, journal = {Advanced Engineering Materials}, keywords = {Ti6Al4V}, number = 21, pages = 2300876, publisher = {Wiley}, title = {On the Reliability of Automated Analysis of Fracture Surfaces Using a Novel Computer Vision-Based Tool}, volume = 25, year = 2023 }
• Active Learning with Fast Model Updates and Class-Balanced Selection for Imbalanced Datasets. Huang, Zhixin; He, Yujiang; Herde, Marek; Huseljic, Denis; Sick, Bernhard. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 28–45. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Active learning strategies aim to minimize the number of queried samples for model training. However, two challenges in pool-based deep active learning on imbalanced datasets are observed in experiments: (1) the declining performance of active learning strategies due to imbalanced class distribution; (2)~the lack of sample diversity in acquisition batches due to the absence of timely model updates. This paper proposes the AL-FaMoUS, a general solution combining fast model updates and class-balanced minibatch selection to the active learning process. Furthermore, a simplification of the AL-FaMoUS, which selects one single sample in each acquisition minibatch, is experimentally evaluated on four image and three time-series imbalanced datasets. The results demonstrate that the implemented AL-FaMoUS outperforms the other adopted AL strategies, including uncertainty sampling and BALD solely combined with either the fast model update or the class balance selection strategy, in terms of accuracy and Macro F1 score.

  @inproceedings{huang2023active, abstract = {Active learning strategies aim to minimize the number of queried samples for model training. However, two challenges in pool-based deep active learning on imbalanced datasets are observed in experiments: (1) the declining performance of active learning strategies due to imbalanced class distribution; (2) the lack of sample diversity in acquisition batches due to the absence of timely model updates. This paper proposes the AL-FaMoUS, a general solution combining fast model updates and class-balanced minibatch selection to the active learning process. Furthermore, a simplification of the AL-FaMoUS, which selects one single sample in each acquisition minibatch, is experimentally evaluated on four image and three time-series imbalanced datasets. The results demonstrate that the implemented AL-FaMoUS outperforms the other adopted AL strategies, including uncertainty sampling and BALD solely combined with either the fast model update or the class balance selection strategy, in terms of accuracy and Macro F1 score.}, author = {Huang, Zhixin and He, Yujiang and Herde, Marek and Huseljic, Denis and Sick, Bernhard}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {ClassBalancedSelection}, pages = {28–45}, title = {Active Learning with Fast Model Updates and Class-Balanced Selection for Imbalanced Datasets}, year = 2023 }
• Resilience of Time-Varying Communication Graphs for Consensus of Changing Sets of Computing Agents. Schmidtke, V.; Liu, Z.; Stursberg, O. In 62nd IEEE Conference on Decision and Control, bll 3474–3479. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  System performance of distributed control systems and networked computing systems is strongly dependent of the underlying communication topology. This paper considers the rarely studied problem of how the topology can maintain resilience by reconfiguration in case that agents leave or join the network during online operation. Existing optimization-based approaches which reconfigure the entire network can typically not be used in this case, since the computational burden for online application is too high. Thus, this paper proposes a novel combined offline-online scheme which optimizes the topology for high convergence rate (of e.g. consensus problems) while providing guarantees for the robustness against agent failures. In the offline part, an optimization of the entire topology is carried out using novel constraints to prepare resilience of the online procedure. For the latter, the proposed scheme guarantees that robustness is maintained for joining agents and if a specified number of agents leave the network. In simulation, the proposed scheme is compared to existing approaches and the advantages of the online-offline procedure are demonstrated.

  @article{schmidtke2023resilience, abstract = {System performance of distributed control systems and networked computing systems is strongly dependent of the underlying communication topology. This paper considers the rarely studied problem of how the topology can maintain resilience by reconfiguration in case that agents leave or join the network during online operation. Existing optimization-based approaches which reconfigure the entire network can typically not be used in this case, since the computational burden for online application is too high. Thus, this paper proposes a novel combined offline-online scheme which optimizes the topology for high convergence rate (of e.g. consensus problems) while providing guarantees for the robustness against agent failures. In the offline part, an optimization of the entire topology is carried out using novel constraints to prepare resilience of the online procedure. For the latter, the proposed scheme guarantees that robustness is maintained for joining agents and if a specified number of agents leave the network. In simulation, the proposed scheme is compared to existing approaches and the advantages of the online-offline procedure are demonstrated.}, author = {Schmidtke, V. and Liu, Z. and Stursberg, O.}, journal = {62nd IEEE Conference on Decision and Control}, keywords = {isac-www}, pages = {3474-3479}, title = {Resilience of Time-Varying Communication Graphs for Consensus of Changing Sets of Computing Agents}, year = 2023 }
• Self-Integration and Agent Compatibility. Gruhl, Christian; Sick, Bernhard. In Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C), bll 71–73. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This position paper addresses the issue of agent compatibility in the context of self-aware microsystems (SAMs), which are SISSY systems with self-awareness capabilities. SAMs are designed to adapt their behaviour and collaborate with other agents in multi-agent settings. Achieving compatibility and effective collaboration between agents pose significant challenges, such as detecting incompatibility and discerning it from malicious behaviour. We present our concept of agent compatibility, highlight future challenges, and propose directions for future research in this area. By focusing on agent compatibility, we aim to advance the understanding of SAMs and promote successful collaboration among agents.

  @inproceedings{gruhl2023self, abstract = {This position paper addresses the issue of agent compatibility in the context of self-aware microsystems (SAMs), which are SISSY systems with self-awareness capabilities. SAMs are designed to adapt their behaviour and collaborate with other agents in multi-agent settings. Achieving compatibility and effective collaboration between agents pose significant challenges, such as detecting incompatibility and discerning it from malicious behaviour. We present our concept of agent compatibility, highlight future challenges, and propose directions for future research in this area. By focusing on agent compatibility, we aim to advance the understanding of SAMs and promote successful collaboration among agents.}, author = {Gruhl, Christian and Sick, Bernhard}, booktitle = {Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)}, keywords = {isac-www}, pages = {71–73}, publisher = {IEEE}, title = {Self-Integration and Agent Compatibility}, year = 2023 }
• Model Predictive Control of PDEs for Temperature Control in 3D-Printing Processes. Schmidtke, V.; Rüger, M.; Stursberg, O. In Proc. of the 2023 European Control Conference, bll 496–501. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Processes of additive manufacturing by 3D-printing are typically operated by only using constant references for feed motion and laser power. For adapting properties of workpieces to desired geometries, the control towards changing reference signals is an enabling factor, though. With a focus on temperature control, this paper proposes a scheme of model predictive control for 3D-printing with consideration of constraints and arbitrary references. Starting from partial differential equations to model heat propagation over the geometry, prediction models and optimization problems are formulated as a local part of a more detailed simulation model of the complete workpiece. For laser velocity and power as manipulated variables, optimization problems involving integer variables are solved. Simulation results show how spatial temperature profiles can be adjusted by the proposed approach.

  @article{schmidtke2023model, abstract = {Processes of additive manufacturing by 3D-printing are typically operated by only using constant references for feed motion and laser power. For adapting properties of workpieces to desired geometries, the control towards changing reference signals is an enabling factor, though. With a focus on temperature control, this paper proposes a scheme of model predictive control for 3D-printing with consideration of constraints and arbitrary references. Starting from partial differential equations to model heat propagation over the geometry, prediction models and optimization problems are formulated as a local part of a more detailed simulation model of the complete workpiece. For laser velocity and power as manipulated variables, optimization problems involving integer variables are solved. Simulation results show how spatial temperature profiles can be adjusted by the proposed approach.}, author = {Schmidtke, V. and Rüger, M. and Stursberg, O.}, journal = {Proc. of the 2023 European Control Conference}, keywords = {isac-www}, pages = {496-501}, title = {Model Predictive Control of PDEs for Temperature Control in 3D-Printing Processes.}, year = 2023 }
• Sampling-based Uncertainty Estimation for an Instance Segmentation Network. Heidecker, Florian; El-Khateeb, Ahmad; Sick, Bernhard. In arXiv e-prints, bl arXiv:2305.14977. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The examination of uncertainty in the predictions of machine learning (ML) models is receiving increasing attention. One uncertainty modeling technique used for this purpose is Monte-Carlo (MC)-Dropout, where repeated predictions are generated for a single input. Therefore, clustering is required to describe the resulting uncertainty, but only through efficient clustering is it possible to describe the uncertainty from the model attached to each object. This article uses Bayesian Gaussian Mixture (BGM) to solve this problem. In addition, we investigate different values for the dropout rate and other techniques, such as focal loss and calibration, which we integrate into the Mask-RCNN model to obtain the most accurate uncertainty approximation of each instance and showcase it graphically.

  @article{heidecker2023sampling, abstract = {The examination of uncertainty in the predictions of machine learning (ML) models is receiving increasing attention. One uncertainty modeling technique used for this purpose is Monte-Carlo (MC)-Dropout, where repeated predictions are generated for a single input. Therefore, clustering is required to describe the resulting uncertainty, but only through efficient clustering is it possible to describe the uncertainty from the model attached to each object. This article uses Bayesian Gaussian Mixture (BGM) to solve this problem. In addition, we investigate different values for the dropout rate and other techniques, such as focal loss and calibration, which we integrate into the Mask-RCNN model to obtain the most accurate uncertainty approximation of each instance and showcase it graphically.}, author = {Heidecker, Florian and El-Khateeb, Ahmad and Sick, Bernhard}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2305.14977}, title = {Sampling-based Uncertainty Estimation for an Instance Segmentation Network}, year = 2023 }
• Active Bird2Vec: Towards End-To-End Bird Sound Monitoring with Transformers. Rauch, Lukas; Schwinger, Raphael; Wirth, Moritz; Sick, Bernhard; Tomforde, Sven; Scholz, Christoph. In Workshop on Artificial Intelligence for Sustainability (AI4S), ECAI, bll 1–6. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ACTIVE BIRD2VEC is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.

  @inproceedings{rauch2023active, abstract = {We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ACTIVE BIRD2VEC is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.}, author = {Rauch, Lukas and Schwinger, Raphael and Wirth, Moritz and Sick, Bernhard and Tomforde, Sven and Scholz, Christoph}, booktitle = {Workshop on Artificial Intelligence for Sustainability (AI4S), ECAI}, keywords = {isac-www}, pages = {1–6}, title = {Active Bird2Vec: Towards End-To-End Bird Sound Monitoring with Transformers}, year = 2023 }
• Reconstruction of incomplete X-ray diffraction pole figures of oligocrystalline materials using deep learning. Meier, David; Ragunathan, Rishan; Degener, Sebastian; Liehr, Alexander; Vollmer, Malte; Niendorf, Thomas; Sick, Bernhard. In Scientific Reports, 13(1), bl 5410. Springer Nature, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  X‑ray diffraction crystallography allows non‑destructive examination of crystal structures. Furthermore, it has low requirements regarding surface preparation, especially compared to electron backscatter diffraction. However, up to now, X‑ray diffraction has been highly time‑ consuming in standard laboratory conditions since intensities on multiple lattice planes have to be recorded by rotating and tilting. Furthermore, examining oligocrystalline materials is challenging due to the limited number of diffraction spots. Moreover, commonly used evaluation methods for crystallographic orientation analysis need multiple lattice planes for a reliable pole figure reconstruction. In this article, we propose a deep‑learning‑based method for oligocrystalline specimens, i.e., specimens with up to three grains of arbitrary crystal orientations. Our approach allows faster experimentation due to accurate reconstructions of pole figure regions, which we did not probe experimentally. In contrast to other methods, the pole figure is reconstructed based on only a single incomplete pole figure. To speed up the development of our proposed method and for usage in other machine learning algorithms, we introduce a GPU‑based simulation for data generation. Furthermore, we present a pole widths standardization technique using a custom deep learning architecture that makes algorithms more robust against influences from the experiment setup and material.

  @article{meier2023reconstruction, abstract = {X‑ray diffraction crystallography allows non‑destructive examination of crystal structures. Furthermore, it has low requirements regarding surface preparation, especially compared to electron backscatter diffraction. However, up to now, X‑ray diffraction has been highly time‑ consuming in standard laboratory conditions since intensities on multiple lattice planes have to be recorded by rotating and tilting. Furthermore, examining oligocrystalline materials is challenging due to the limited number of diffraction spots. Moreover, commonly used evaluation methods for crystallographic orientation analysis need multiple lattice planes for a reliable pole figure reconstruction. In this article, we propose a deep‑learning‑based method for oligocrystalline specimens, i.e., specimens with up to three grains of arbitrary crystal orientations. Our approach allows faster experimentation due to accurate reconstructions of pole figure regions, which we did not probe experimentally. In contrast to other methods, the pole figure is reconstructed based on only a single incomplete pole figure. To speed up the development of our proposed method and for usage in other machine learning algorithms, we introduce a GPU‑based simulation for data generation. Furthermore, we present a pole widths standardization technique using a custom deep learning architecture that makes algorithms more robust against influences from the experiment setup and material.}, author = {Meier, David and Ragunathan, Rishan and Degener, Sebastian and Liehr, Alexander and Vollmer, Malte and Niendorf, Thomas and Sick, Bernhard}, journal = {Scientific Reports}, keywords = {isac-www}, number = 1, pages = 5410, publisher = {Springer Nature}, title = {Reconstruction of incomplete {X}-ray diffraction pole figures of oligocrystalline materials using deep learning}, volume = 13, year = 2023 }
• Model-Based Optimization of Vaccination Strategies in Different Phases of Pandemic Virus Spread. Liu, Z.; Omayrat, M.; Stursberg, O. In Informatics in Control, Automation and Robotics. ICINCO 2021. Lecture Notes in Electrical Engineering, 1006, bll 185–208. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper aims at applying optimal control principles to investigate optimal vaccination strategies in different phases of a pandemic. Background of the study is that many countries have started their vaccination procedures against the COVID-19 disease in the beginning of 2021, but supply shortages for the vaccines prevented that everyone could be vaccinated immediately. At the beginning of 2022, in contrast, the vaccine supply was ample, but the effectiveness of different existing vaccines to avoid infection by new virus variants was in doubt, as well as the acceptance of booster doses decreased over time. To account for these effects, two formulations of optimization tasks based on different epidemic models are proposed in this paper. The solution of these tasks determines optimal distribution strategies for available vaccines, and optimized vaccination schemes to reduce the need of booster doses for later phase. Effectiveness of these strategies compared with other popular strategies (as applied in practice) is demonstrated through a series of simulations.

  @article{liu2023modelbased, abstract = {This paper aims at applying optimal control principles to investigate optimal vaccination strategies in different phases of a pandemic. Background of the study is that many countries have started their vaccination procedures against the COVID-19 disease in the beginning of 2021, but supply shortages for the vaccines prevented that everyone could be vaccinated immediately. At the beginning of 2022, in contrast, the vaccine supply was ample, but the effectiveness of different existing vaccines to avoid infection by new virus variants was in doubt, as well as the acceptance of booster doses decreased over time. To account for these effects, two formulations of optimization tasks based on different epidemic models are proposed in this paper. The solution of these tasks determines optimal distribution strategies for available vaccines, and optimized vaccination schemes to reduce the need of booster doses for later phase. Effectiveness of these strategies compared with other popular strategies (as applied in practice) is demonstrated through a series of simulations.}, author = {Liu, Z. and Omayrat, M. and Stursberg, O.}, journal = {Informatics in Control, Automation and Robotics. ICINCO 2021. Lecture Notes in Electrical Engineering}, keywords = {isac-www}, pages = {185–208}, title = {Model-Based Optimization of Vaccination Strategies in Different Phases of Pandemic Virus Spread}, volume = 1006, year = 2023 }
• A Digital Twin for Part QualityPrediction and Control in Plastic Injection Molding. Rehmer, Alexander; Klute, Marco; Kroll, Andreas; Heim, Hans-Peter. In Modeling, Identification, and Control for Cyber-Physical Systems Towards Industry 4.0, 1st , P. Mercorelli, W. Zhang, H. Nemati, Y. Zhang (reds.). Elsevier, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inbook{EmergMeth_Rehmer_2022, author = {Rehmer, Alexander and Klute, Marco and Kroll, Andreas and Heim, Hans-Peter}, booktitle = {Modeling, Identification, and Control for Cyber-Physical Systems Towards Industry 4.0}, edition = {1st}, editor = {Mercorelli, Paolo and Zhang, Weicun and Nemati, Hamidreza and Zhang, YuMing}, keywords = {isac-www}, note = {accepted}, publisher = {Elsevier}, series = {Emerging Methodologies and Applications in Modelling, Identification and Control}, title = {A Digital Twin for Part QualityPrediction and Control in Plastic Injection Molding}, year = 2023 }
• Spatio-temporal LPV model of 2D workpiece temperature for Direct Laser Deposition. Pereira, Guilherme; Jelicic, Goran; Kroll, Andreas. In 22nd IFAC World Congress. IFAC, Yokohama, 2023.
  • [ BibTeX ]
  @inproceedings{Pereira-IFAC-2023, address = {Yokohama}, author = {Pereira, Guilherme and Jelicic, Goran and Kroll, Andreas}, booktitle = {22nd IFAC World Congress}, keywords = {isac-www}, month = {Juli}, note = {accepted}, organization = {IFAC}, title = {Spatio-temporal LPV model of 2D workpiece temperature for Direct Laser Deposition}, year = 2023 }
• Integration of a High-Precision 3D Sensor into a 3D Thermography System. Mendez, Miguel; Schramm, Sebastian; Schmoll, Robert; Kroll, Andreas. In Sensor and Measurement Science International Conference (SMSI) 2023. AMA Verband für Sensorik und Messtechnik e.V., Nürnberg, 2023.
  • [ BibTeX ]
  @inproceedings{Mendez2023a, address = {Nürnberg}, author = {Mendez, Miguel and Schramm, Sebastian and Schmoll, Robert and Kroll, Andreas}, booktitle = {Sensor and Measurement Science International Conference (SMSI) 2023}, keywords = {isac-www}, month = {May}, organization = {AMA Verband für Sensorik und Messtechnik e.V.}, title = {Integration of a High-Precision 3D Sensor into a 3D Thermography System}, year = 2023 }
• An Airborne Measurement System to Detect, Locate and Quantify Methane Emissions. Kistner, Lars; Schmoll, Robert; Kroll, Andreas. In Sensor and Measurement Science International Conference (SMSI) 2023. AMA Verband für Sensorik und Messtechnik e.V., Nürnberg, 2023.
  • [ BibTeX ]
  @inproceedings{Kistner2023, address = {Nürnberg}, author = {Kistner, Lars and Schmoll, Robert and Kroll, Andreas}, booktitle = {Sensor and Measurement Science International Conference (SMSI) 2023}, keywords = {isac-www}, month = {May}, organization = {AMA Verband für Sensorik und Messtechnik e.V.}, title = {An Airborne Measurement System to Detect, Locate and Quantify Methane Emissions}, year = 2023 }
• Utilizing Continuous Kernels for Processing Irregularly and Inconsistently Sampled Data With Position-Dependent Features. Magnussen, Birk Martin; Stern, Claudius; Sick, Bernhard. In International Conference on Autonomic and Autonomous Systems (ICAS), bll 49–53. ThinkMind, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Continuous Kernels have been a recent development in convolutional neural networks. Such kernels are used to process data sampled at different resolutions as well as irregularly and inconsistently sampled data. Convolutional neural networks have the property of translational invariance (e.g., features are detected regardless of their position in the measurement domain), which is unsuitable for certain types of data, where the position of detected features is relevant. However, the capabilities of continuous kernels to process irregularly sampled data are still desired. This article introduces a novel method utilizing continuous kernels for detecting global features at absolute positions in the data domain. Through a use case in processing multiple spatially resolved reflection spectroscopy data, which is sampled irregularly and inconsistently, we show that the proposed method is capable of processing such data natively without additional preprocessing as is needed using comparable methods. In addition, we show that the proposed method is able to achieve a higher prediction accuracy than a comparable network on a dataset with position-dependent features. Furthermore, a higher robustness to missing data compared to a benchmark network using data interpolation is observed, which allows the network to adapt to sensors with individual failed components without the need for retraining.

  @inproceedings{magnussen2023utilizing, abstract = {Continuous Kernels have been a recent development in convolutional neural networks. Such kernels are used to process data sampled at different resolutions as well as irregularly and inconsistently sampled data. Convolutional neural networks have the property of translational invariance (e.g., features are detected regardless of their position in the measurement domain), which is unsuitable for certain types of data, where the position of detected features is relevant. However, the capabilities of continuous kernels to process irregularly sampled data are still desired. This article introduces a novel method utilizing continuous kernels for detecting global features at absolute positions in the data domain. Through a use case in processing multiple spatially resolved reflection spectroscopy data, which is sampled irregularly and inconsistently, we show that the proposed method is capable of processing such data natively without additional preprocessing as is needed using comparable methods. In addition, we show that the proposed method is able to achieve a higher prediction accuracy than a comparable network on a dataset with position-dependent features. Furthermore, a higher robustness to missing data compared to a benchmark network using data interpolation is observed, which allows the network to adapt to sensors with individual failed components without the need for retraining.}, author = {Magnussen, Birk Martin and Stern, Claudius and Sick, Bernhard}, booktitle = {International Conference on Autonomic and Autonomous Systems (ICAS)}, keywords = {ReflectionSpectroscopy}, pages = {49–53}, publisher = {ThinkMind}, title = {Utilizing Continuous Kernels for Processing Irregularly and Inconsistently Sampled Data With Position-Dependent Features}, year = 2023 }
• Reference Tracking for Constrained Uncertain Linear Systems by Stochastic MPC. Hahn, J.; Stursberg, O. In Proc. of the 22nd IFAC World Congress, 56(2), bll 10421–10427. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper addresses model predictive control of a class of linear systems subject to additive stochastic disturbances and constraints. The underlying stochastic optimal control problem combines inverse cumulative distribution functions with ellipsoid-in-polyhedron formulations to reduce the conservatism induced by constraint satisfaction. By use of terminal constraints and time-varying weights within the cost functional, the presented control scheme satisfies criteria for mean-square stability and can be adapted to reference-tracking problems for arbitrary reference signals.

  @article{hahn2023reference, abstract = {This paper addresses model predictive control of a class of linear systems subject to additive stochastic disturbances and constraints. The underlying stochastic optimal control problem combines inverse cumulative distribution functions with ellipsoid-in-polyhedron formulations to reduce the conservatism induced by constraint satisfaction. By use of terminal constraints and time-varying weights within the cost functional, the presented control scheme satisfies criteria for mean-square stability and can be adapted to reference-tracking problems for arbitrary reference signals.}, author = {Hahn, J. and Stursberg, O.}, journal = {Proc. of the 22nd IFAC World Congress}, keywords = {isac-www}, number = 2, pages = {10421-10427}, title = {Reference Tracking for Constrained Uncertain Linear Systems by Stochastic MPC}, volume = 56, year = 2023 }
• Graph Neural Networks Designed for Different Graph Types: A Survey. Thomas, Josephine; Moallemy-Oureh, Alice; Beddar-Wiesing, Silvia; Holzhüter, Clara. In Transactions on Machine Learning Research. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Graphs are ubiquitous in nature and can therefore serve as models for many practical but also theoretical problems. For this purpose, they can be defined as many different types which suitably reflect the individual contexts of the represented problem. To address cutting-edge problems based on graph data, the research field of Graph Neural Networks (GNNs) has emerged. Despite the field’s youth and the speed at which new models are developed, many recent surveys have been published to keep track of them. Nevertheless, it has not yet been gathered which GNN can process what kind of graph types. In this survey, we give a detailed overview of already existing GNNs and, unlike previous surveys, categorize them according to their ability to handle different graph types and properties. We consider GNNs operating on static and dynamic graphs of different structural constitutions, with or without node or edge attributes. Moreover, we distinguish between GNN models for discrete-time or continuous-time dynamic graphs and group the models according to their architecture. We find that there are still graph types that are not or only rarely covered by existing GNN models. We point out where models are missing and give potential reasons for their absence.

  @article{thomas2023graph, abstract = {Graphs are ubiquitous in nature and can therefore serve as models for many practical but also theoretical problems. For this purpose, they can be defined as many different types which suitably reflect the individual contexts of the represented problem. To address cutting-edge problems based on graph data, the research field of Graph Neural Networks (GNNs) has emerged. Despite the field’s youth and the speed at which new models are developed, many recent surveys have been published to keep track of them. Nevertheless, it has not yet been gathered which GNN can process what kind of graph types. In this survey, we give a detailed overview of already existing GNNs and, unlike previous surveys, categorize them according to their ability to handle different graph types and properties. We consider GNNs operating on static and dynamic graphs of different structural constitutions, with or without node or edge attributes. Moreover, we distinguish between GNN models for discrete-time or continuous-time dynamic graphs and group the models according to their architecture. We find that there are still graph types that are not or only rarely covered by existing GNN models. We point out where models are missing and give potential reasons for their absence.}, author = {Thomas, Josephine and Moallemy-Oureh, Alice and Beddar-Wiesing, Silvia and Holzhüter, Clara}, journal = {Transactions on Machine Learning Research}, keywords = {graph_neural_network}, title = {Graph Neural Networks Designed for Different Graph Types: A Survey}, year = 2023 }
• Distributed MPC of Uncertain Multi-Agent Systems Considering Formations and Obstacles. Flüs, P.; Stursberg, O. In Proc. of the 22nd IFAC World Congress, Volume 56(Issue 2), bll 10155–10161. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper proposes a method to control a class of multi-agent systems with uncertainties modeled as stochastic processes with arbitrary probability distributions. The considered control problem is to lead a formation of agents through a space which is partially obstructed by obstacles. The proposed solution is to use a hierarchically structured approach of distributed stochastic model predictive control (DSMPC). The approach combines elements of formation reference structures, leader-follower concepts, and successive convexification (SC) for collision avoidance. To consider the stochastic uncertainties, over-approximated probabilistic reachable sets (PRS) are computed based on Chebyshev's inequality. The nominal (expected) agent behavior is optimized within the DSMPC such that (probabilistic) constraints are satisfied in a distributed way. For the overall approach, closed-loop stability of the distributed control concept is investigated and an illustrating example is provided.

  @article{flus2023distributed, abstract = {This paper proposes a method to control a class of multi-agent systems with uncertainties modeled as stochastic processes with arbitrary probability distributions. The considered control problem is to lead a formation of agents through a space which is partially obstructed by obstacles. The proposed solution is to use a hierarchically structured approach of distributed stochastic model predictive control (DSMPC). The approach combines elements of formation reference structures, leader-follower concepts, and successive convexification (SC) for collision avoidance. To consider the stochastic uncertainties, over-approximated probabilistic reachable sets (PRS) are computed based on Chebyshev's inequality. The nominal (expected) agent behavior is optimized within the DSMPC such that (probabilistic) constraints are satisfied in a distributed way. For the overall approach, closed-loop stability of the distributed control concept is investigated and an illustrating example is provided.}, author = {Flüs, P. and Stursberg, O.}, journal = {Proc. of the 22nd IFAC World Congress}, keywords = {isac-www}, number = {Issue 2}, pages = {10155-10161}, title = {Distributed MPC of Uncertain Multi-Agent Systems Considering Formations and Obstacles}, volume = {Volume 56}, year = 2023 }
• On Optimal Synchronization of Diffusively Coupled Heterogeneous Van Der Pol Oscillators. Trummel, T.; Liu, Z.; Stursberg, O. In IFAC-PapersOnLine, Volume 56(Issue 2), bll 9475–9480. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper proposes a novel method to achieve and preserve synchronization for a set of connected heterogeneous Van der Pol oscillators. Unlike the state-of-the-art synchronization methods, in which a large coupling gain is applied to couple any pair of connected oscillators, the proposed method first casts the whole synchronization process into two phases. The first one considers the period from the beginning to the first instant of synchronization, while the second phase covers the following time in which synchronization must be preserved. It is shown that a large coupling gain is adopted for the first phase, while the averaged coupling gain to preserve the synchronization in the second phase can be reduced significantly by using an offline optimized coupling law. Efficiency and performance of this method are confirmed by a set of numerical tests with different graphs and system dynamics.

  @article{trummel2023optimal, abstract = {This paper proposes a novel method to achieve and preserve synchronization for a set of connected heterogeneous Van der Pol oscillators. Unlike the state-of-the-art synchronization methods, in which a large coupling gain is applied to couple any pair of connected oscillators, the proposed method first casts the whole synchronization process into two phases. The first one considers the period from the beginning to the first instant of synchronization, while the second phase covers the following time in which synchronization must be preserved. It is shown that a large coupling gain is adopted for the first phase, while the averaged coupling gain to preserve the synchronization in the second phase can be reduced significantly by using an offline optimized coupling law. Efficiency and performance of this method are confirmed by a set of numerical tests with different graphs and system dynamics.}, author = {Trummel, T. and Liu, Z. and Stursberg, O.}, journal = {IFAC-PapersOnLine}, keywords = {isac-www}, number = {Issue 2}, pages = {9475-9480}, title = {On Optimal Synchronization of Diffusively Coupled Heterogeneous Van Der Pol Oscillators}, volume = {Volume 56}, year = 2023 }
• Distributed Interval Estimation for Continuous-Time Linear Systems: A Two-Step Method. Wang, Z.; Zhang, J.; Liu, Z.; Han, W.; Shen, M. In Proc. of the 22nd IFAC World Congress, Volume 56(Issue 2), bll 8500–8505. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper proposes a two-step distributed interval estimation method for continuous-time linear time-invariant (LTI) systems subject to unknown but bounded disturbance and measurement noise. First, a network of local observers is designed to obtain point-wise estimation. Second, interval estimation is obtained based on the point-wise estimation and peak-to-peak analysis. The conditions of the observer design and the peak-to-peak analysis are formulated as linear matrix inequalities, which can be solved efficiently. Compared with the distributed interval observers designed using the monotone system theory, the proposed method has less restrictive design conditions and hence broader application scopes. Moreover, the parameter matrices in the proposed method can be optimized to obtain tight interval estimation. A numerical simulation is given to illustrate the performance of the proposed method.

  @article{wang2023distributed, abstract = {This paper proposes a two-step distributed interval estimation method for continuous-time linear time-invariant (LTI) systems subject to unknown but bounded disturbance and measurement noise. First, a network of local observers is designed to obtain point-wise estimation. Second, interval estimation is obtained based on the point-wise estimation and peak-to-peak analysis. The conditions of the observer design and the peak-to-peak analysis are formulated as linear matrix inequalities, which can be solved efficiently. Compared with the distributed interval observers designed using the monotone system theory, the proposed method has less restrictive design conditions and hence broader application scopes. Moreover, the parameter matrices in the proposed method can be optimized to obtain tight interval estimation. A numerical simulation is given to illustrate the performance of the proposed method.}, author = {Wang, Z. and Zhang, J. and Liu, Z. and Han, W. and Shen, M.}, journal = {Proc. of the 22nd IFAC World Congress}, keywords = {isac-www}, number = {Issue 2}, pages = {8500-8505}, title = {Distributed Interval Estimation for Continuous-Time Linear Systems: A Two-Step Method.}, volume = {Volume 56}, year = 2023 }
• Tailored Output Layers of Neural Networks for Satisfaction of State Constraints in Nonlinear Control Systems. Markolf, L.; Stursberg, O. In Proceedings of the 2023 Annual American Control Conference (ACC), bll 1881–1888. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  This work considers the synthesis of state feedback controllers established as deep artificial feed-forward neural networks for the control of discrete-time nonlinear but input-affine systems. The idea is to design output layers of particular structure to guarantee the satisfaction of state constraints in form of control-invariant ellipsoids. Since an analytical expression can be derived for the resulting neural network controller, the latter can be stored and evaluated efficiently. Moreover, the proposed output layer guarantees the satisfaction of the considered state constraints for each specification of the parameter vector. Numerical examples are provided for illustration and evaluation of the approach, in which the approximation of a nonlinear model predictive control law is considered as application.

  @article{markolf2023tailored, abstract = {This work considers the synthesis of state feedback controllers established as deep artificial feed-forward neural networks for the control of discrete-time nonlinear but input-affine systems. The idea is to design output layers of particular structure to guarantee the satisfaction of state constraints in form of control-invariant ellipsoids. Since an analytical expression can be derived for the resulting neural network controller, the latter can be stored and evaluated efficiently. Moreover, the proposed output layer guarantees the satisfaction of the considered state constraints for each specification of the parameter vector. Numerical examples are provided for illustration and evaluation of the approach, in which the approximation of a nonlinear model predictive control law is considered as application.}, author = {Markolf, L. and Stursberg, O.}, journal = {Proceedings of the 2023 Annual American Control Conference (ACC)}, keywords = {isac-www}, pages = {1881-1888}, title = {Tailored Output Layers of Neural Networks for Satisfaction of State Constraints in Nonlinear Control Systems}, year = 2023 }
• DNN-Approximations of Control Laws for Nonlinear Systems with Polytopic Constraints. Markolf, L.; Stursberg, O. In Proc. of the 22nd IFAC World Congress, bll 1516–1521. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  This work focuses on deep artificial feed-forward neural networks as parametric function approximators in optimal control of discrete-time nonlinear but affine-in-control systems subject to polytopic constraints on the continuous states and control inputs. The neural networks are either considered as approximators of optimal control laws or as approximators of the corresponding cost functions. In both cases, an approach is developed for determining the approximated optimal control inputs without violating the constraints. A simple approximate policy iteration algorithm exploiting both approaches is finally presented and illustrated for a numerical example.

  @article{markolf2023dnnapproximations, abstract = {This work focuses on deep artificial feed-forward neural networks as parametric function approximators in optimal control of discrete-time nonlinear but affine-in-control systems subject to polytopic constraints on the continuous states and control inputs. The neural networks are either considered as approximators of optimal control laws or as approximators of the corresponding cost functions. In both cases, an approach is developed for determining the approximated optimal control inputs without violating the constraints. A simple approximate policy iteration algorithm exploiting both approaches is finally presented and illustrated for a numerical example.}, author = {Markolf, L. and Stursberg, O.}, journal = {Proc. of the 22nd IFAC World Congress}, keywords = {isac-www}, pages = {1516-1521}, title = {DNN-Approximations of Control Laws for Nonlinear Systems with Polytopic Constraints.}, year = 2023 }
• Power flow forecasts at transmission grid nodes using Graph Neural Networks. Beinert, Dominik; Holzhüter, Clara; Thomas, Josephine; Vogt, Stephan. In Energy and AI, 14(1), bl 100262. Elsevier, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The increasing share of renewable energy in the electricity grid and progressing changes in power consumption have led to fluctuating, and weather-dependent power flows. To ensure grid stability, grid operators rely on power forecasts which are crucial for grid calculations and planning. In this paper, a Multi-Task Learning approach is combined with a Graph Neural Network (GNN) to predict vertical power flows at transformers connecting high and extra-high voltage levels. The proposed method accounts for local differences in power flow characteristics by using an Embedding Multi-Task Learning approach. The use of a Bayesian embedding to capture the latent node characteristics allows to share the weights across all transformers in the subsequent node-invariant GNN while still allowing the individual behavioral patterns of the transformers to be distinguished. At the same time, dependencies between transformers are considered by the GNN architecture which can learn relationships between different transformers and thus take into account that power flows in an electricity network are not independent from each other. The effectiveness of the proposed method is demonstrated through evaluation on two real-world data sets provided by two of four German Transmission System Operators, comprising large portions of the operated German transmission grid. The results show that the proposed Multi-Task Graph Neural Network is a suitable representation learner for electricity networks with a clear advantage provided by the preceding embedding layer. It is able to capture interconnections between correlated transformers and indeed improves the performance in power flow prediction compared to standard Neural Networks. A sign test shows that the proposed model reduces the test RMSE on both data sets compared to the benchmark models significantly.

  @article{beinert2023power, abstract = {The increasing share of renewable energy in the electricity grid and progressing changes in power consumption have led to fluctuating, and weather-dependent power flows. To ensure grid stability, grid operators rely on power forecasts which are crucial for grid calculations and planning. In this paper, a Multi-Task Learning approach is combined with a Graph Neural Network (GNN) to predict vertical power flows at transformers connecting high and extra-high voltage levels. The proposed method accounts for local differences in power flow characteristics by using an Embedding Multi-Task Learning approach. The use of a Bayesian embedding to capture the latent node characteristics allows to share the weights across all transformers in the subsequent node-invariant GNN while still allowing the individual behavioral patterns of the transformers to be distinguished. At the same time, dependencies between transformers are considered by the GNN architecture which can learn relationships between different transformers and thus take into account that power flows in an electricity network are not independent from each other. The effectiveness of the proposed method is demonstrated through evaluation on two real-world data sets provided by two of four German Transmission System Operators, comprising large portions of the operated German transmission grid. The results show that the proposed Multi-Task Graph Neural Network is a suitable representation learner for electricity networks with a clear advantage provided by the preceding embedding layer. It is able to capture interconnections between correlated transformers and indeed improves the performance in power flow prediction compared to standard Neural Networks. A sign test shows that the proposed model reduces the test RMSE on both data sets compared to the benchmark models significantly.}, author = {Beinert, Dominik and Holzhüter, Clara and Thomas, Josephine and Vogt, Stephan}, journal = {Energy and AI}, keywords = {isac-www}, number = 1, pages = 100262, publisher = {Elsevier}, title = {Power flow forecasts at transmission grid nodes using Graph Neural Networks}, volume = 14, year = 2023 }
• Self-awareness in Cyber-Physical Systems: Recent Developments and Open Challenges. Esterle, Lukas; Dutt, Nikil; Gruhl, Christian; Lewis, Peter R.; Marcenaro, Lucio; Regazzoni, Carlo; Jantsch, Axel. In Design, Automation & Test in Europe Conference & Exhibition (DATE), bll 1–6. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Self-aware computing systems enable computing systems to reflect on their actions and behavior. This becomes even more relevant in Cyber-Physical Systems where computing systems have to control and interact with elements in the real world. This paper reports on recent advances made in computational self-awareness for cyber-physical systems.

  @inproceedings{esterle2023self, abstract = {Self-aware computing systems enable computing systems to reflect on their actions and behavior. This becomes even more relevant in Cyber-Physical Systems where computing systems have to control and interact with elements in the real world. This paper reports on recent advances made in computational self-awareness for cyber-physical systems.}, author = {Esterle, Lukas and Dutt, Nikil and Gruhl, Christian and Lewis, Peter R. and Marcenaro, Lucio and Regazzoni, Carlo and Jantsch, Axel}, booktitle = {Design, Automation & Test in Europe Conference & Exhibition (DATE)}, keywords = {isac-www}, pages = {1–6}, publisher = {IEEE}, title = {Self-awareness in Cyber-Physical Systems: Recent Developments and Open Challenges}, year = 2023 }
• ActiveGLAE: A Benchmark for Deep Active Learning with Transformers. Rauch, Lukas; Aßenmacher, Matthias; Huseljic, Denis; Wirth, Moritz; Bischl, Bernd; Sick, Bernhard. In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD), bll 55–74. Springer, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is cur- rently no standardized evaluation protocol for transformer-based lan- guage models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for prac- titioners. To tackle this challenge, we propose the ActiveGLAE bench- mark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.

  @inproceedings{rauch2023activeglae, abstract = {Deep active learning (DAL) seeks to reduce annotation costs by enabling the model to actively query instance annotations from which it expects to learn the most. Despite extensive research, there is cur- rently no standardized evaluation protocol for transformer-based lan- guage models in the field of DAL. Diverse experimental settings lead to difficulties in comparing research and deriving recommendations for prac- titioners. To tackle this challenge, we propose the ActiveGLAE bench- mark, a comprehensive collection of data sets and evaluation guidelines for assessing DAL. Our benchmark aims to facilitate and streamline the evaluation process of novel DAL strategies. Additionally, we provide an extensive overview of current practice in DAL with transformer-based language models. We identify three key challenges - data set selection, model training, and DAL settings - that pose difficulties in comparing query strategies. We establish baseline results through an extensive set of experiments as a reference point for evaluating future work. Based on our findings, we provide guidelines for researchers and practitioners.}, author = {Rauch, Lukas and Aßenmacher, Matthias and Huseljic, Denis and Wirth, Moritz and Bischl, Bernd and Sick, Bernhard}, booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD)}, keywords = {isac-www}, pages = {55–74}, publisher = {Springer}, title = {ActiveGLAE: A Benchmark for Deep Active Learning with Transformers}, year = 2023 }
• Distributed conditional cooperation model predictive control of interconnected microgrids. Sampathirao, Ajay Kumar; Hofmann, Steffen; Raisch, Jörg; Hans, Christian Andreas. In Automatica, 157, bl 111258. Elsevier BV, 2023.
  • [ BibTeX ]
  • [ URL ]
  @article{Sampathirao_2023, author = {Sampathirao, Ajay Kumar and Hofmann, Steffen and Raisch, Jörg and Hans, Christian Andreas}, journal = {Automatica}, keywords = {isac-www}, month = {nov}, pages = 111258, publisher = {Elsevier BV}, title = {Distributed conditional cooperation model predictive control of interconnected microgrids}, volume = 157, year = 2023 }
• Comparative Assessment of the Size-of-Source Effect in Middle and Long-Wavelength Infrared Cameras. Mendez, Miguel; Ebert, Jannik; Sommerlade, Lars; Schmoll, Robert; Kroll, Andreas. In QIRT Asia 2023 - The 4th Quantitative Infrared Thermography Conference. Abu Dhabi, U.A.E., 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Current radiometric calibration procedures for thermal imaging cameras are usually performed for a single calibration geometry. The uncertainty estimated from such procedure only represents the calibration scenario and does not account for variation of the emitting source&amp;#x27;s size; this is known as the size-of-source effect (SSE). In this work, the measurements of the SSE for four thermal imaging cameras (two in the long-wavelength and two in the middle-wavelength infrared range) are presented. The investigation shows that the SSE is smaller for the middle-wavelength cameras.

  @inproceedings{Mendez2023b, abstract = {Current radiometric calibration procedures for thermal imaging cameras are usually performed for a single calibration geometry. The uncertainty estimated from such procedure only represents the calibration scenario and does not account for variation of the emitting source\&\#x27;s size; this is known as the size-of-source effect (SSE). In this work, the measurements of the SSE for four thermal imaging cameras (two in the long-wavelength and two in the middle-wavelength infrared range) are presented. The investigation shows that the SSE is smaller for the middle-wavelength cameras.}, address = {Abu Dhabi, U.A.E.}, author = {Mendez, Miguel and Ebert, Jannik and Sommerlade, Lars and Schmoll, Robert and Kroll, Andreas}, booktitle = {QIRT Asia 2023 - The 4th Quantitative Infrared Thermography Conference}, keywords = {isac-www}, month = {Oct. 30 - Nov. 3}, title = {Comparative Assessment of the Size-of-Source Effect in Middle and Long-Wavelength Infrared Cameras}, year = 2023 }
• Marked Neural Spatio-Temporal Point Process Involving a Dynamic Graph Neural Network. Moallemy-Oureh, Alice; Beddar-Wiesing, Silvia; Nather, Rüdiger; Thomas, Josephine. In Workshop on Temporal Graph Learning (TGL), NeurIPS, bll 1–7. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Temporal Point Processes (TPPs) have recently become increasingly interesting for learning dynamics in graph data. A reason for this is that learning on dynamic graph data is becoming more relevant, since data from many scientific fields, ranging from mathematics, biology, social sciences, and physics to computer science, is naturally related and inherently dynamic. In addition, TPPs provide a meaningful characterization of event streams and a prediction mechanism for future events. Therefore, (semi-)parameterized Neural TPPs have been introduced whose characterization can be (partially) learned and, thus, enable the representation of more complex phenomena. However, the research on modeling dynamic graphs with TPPs is relatively young, and only a few models for node attribute changes or evolving edges have been proposed yet. To allow for learning on fully dynamic graph streams, i.e., graphs that can change in their structure (addition/deletion of nodes/edge) and in their node/edge attributes, we propose a Marked Neural Spatio-Temporal Point Process (MNSTPP). It leverages a Dynamic Graph Neural Network to learn a Marked TPP that handles attributes and spatial data to model and predict any event in a graph stream.

  @inproceedings{moallemyoureh2023marked, abstract = {Temporal Point Processes (TPPs) have recently become increasingly interesting for learning dynamics in graph data. A reason for this is that learning on dynamic graph data is becoming more relevant, since data from many scientific fields, ranging from mathematics, biology, social sciences, and physics to computer science, is naturally related and inherently dynamic. In addition, TPPs provide a meaningful characterization of event streams and a prediction mechanism for future events. Therefore, (semi-)parameterized Neural TPPs have been introduced whose characterization can be (partially) learned and, thus, enable the representation of more complex phenomena. However, the research on modeling dynamic graphs with TPPs is relatively young, and only a few models for node attribute changes or evolving edges have been proposed yet. To allow for learning on fully dynamic graph streams, i.e., graphs that can change in their structure (addition/deletion of nodes/edge) and in their node/edge attributes, we propose a Marked Neural Spatio-Temporal Point Process (MNSTPP). It leverages a Dynamic Graph Neural Network to learn a Marked TPP that handles attributes and spatial data to model and predict any event in a graph stream.}, author = {Moallemy-Oureh, Alice and Beddar-Wiesing, Silvia and Nather, Rüdiger and Thomas, Josephine}, booktitle = {Workshop on Temporal Graph Learning (TGL), NeurIPS}, keywords = {isac-www}, pages = {1–7}, title = {Marked Neural Spatio-Temporal Point Process Involving a Dynamic Graph Neural Network}, year = 2023 }
• Measurement of Fluid Flow Velocity by Using Infrared and Visual Cameras: Comparison and Evaluation of Optical Flow Estimation Algorithms. Shen, Zhe; Schmoll, Robert; Kroll, Andreas. In IEEE Sensors 2023. IEEE, Vienna, Austria, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{Shen2023, address = {Vienna, Austria}, author = {Shen, Zhe and Schmoll, Robert and Kroll, Andreas}, booktitle = {IEEE Sensors 2023}, keywords = {isac-www}, month = {30. October}, organization = {IEEE}, title = {Measurement of Fluid Flow Velocity by Using Infrared and Visual Cameras: Comparison and Evaluation of Optical Flow Estimation Algorithms}, year = 2023 }
• Holistic Modeling of Ultra-High Performance Concrete Production Process: Synergizing Mix Design Fresh Concrete Properties and Curing Conditions. Rezazadeh, Farzad; Dürrbaum, Axel; Zimmermann, Gregor; Kroll, Andreas. In 33. Workshop Computational Intelligence, bll 215–237. KIT Scientific Publishing, Berlin, Germany, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{RezazadehGMA2023, address = {Berlin, Germany}, author = {Rezazadeh, Farzad and Dürrbaum, Axel and Zimmermann, Gregor and Kroll, Andreas}, booktitle = {33. Workshop Computational Intelligence}, keywords = {isac-www}, month = {23.–24. November}, pages = {215–237}, publisher = {KIT Scientific Publishing}, title = {Holistic Modeling of Ultra-High Performance Concrete Production Process: Synergizing Mix Design Fresh Concrete Properties and Curing Conditions}, year = 2023 }
• Automatic Camera-based advanced Slump FlowTesting for Improved Reliability. Dürrbaum, Axel; Rezazadeh, Farzad; Kroll, Andreas. In IEEE Sensors 2023. IEEE, Vienna, Austria, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{2023-ad_fr_ak_-Senors_2023-SFT_Camera, address = {Vienna, Austria}, author = {Dürrbaum, Axel and Rezazadeh, Farzad and Kroll, Andreas}, booktitle = {IEEE Sensors 2023}, keywords = {isac-www}, month = {30. October}, organization = {IEEE}, title = {Automatic Camera-based advanced Slump FlowTesting for Improved Reliability}, year = 2023 }
• Data driven Structure Identification of Takagi Sugeno Fuzzy Models Using a Bounded Error Approach. Wittich, Felix; Kroll, Andreas. In Proceedings of the IEEE International Conference on Fuzzy Systems. Songdo Incheon, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{Wittich-FUZZ-2023, address = {Songdo Incheon}, author = {Wittich, Felix and Kroll, Andreas}, booktitle = {Proceedings of the IEEE International Conference on Fuzzy Systems}, keywords = {isac-www}, month = {August}, title = {Data driven Structure Identification of Takagi Sugeno Fuzzy Models Using a Bounded Error Approach}, year = 2023 }
• Spatio-temporal LPV model of 2D workpiece temperature for Direct Laser Deposition. da Fonseca Pereira, Guilherme; Jelicic, Goran; Kroll, Andreas. In IFAC-PapersOnLine, 56(2), bll 6606–6611. 2023.
  • [ BibTeX ]
  • [ URL ]
  @article{Pereira-IFAC-2023, author = {da Fonseca Pereira, Guilherme and Jelicic, Goran and Kroll, Andreas}, journal = {IFAC-PapersOnLine}, keywords = {isac-www}, month = {Juli}, number = 2, pages = {6606–6611}, series = {22nd IFAC World Congress}, title = {Spatio-temporal {LPV} model of {2D} workpiece temperature for Direct Laser Deposition}, volume = 56, year = 2023 }
• Zum Einsatz von KI-Methoden zur Lösung von Problemen der Regelungstechnik. Sommer, Hanns; Kroll, Andreas. In 33. Workshop Computational Intelligence, bll 61–80. KIT Scientific Publishing, Berlin, Germany, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{2023-HJS_AK-Workshop_CI-KI_RT, address = {Berlin, Germany}, author = {Sommer, Hanns and Kroll, Andreas}, booktitle = {33. Workshop Computational Intelligence}, keywords = {isac-www}, month = {23.–24. November}, organization = {Workshop CI}, pages = {61–80}, publisher = {KIT Scientific Publishing}, title = {Zum Einsatz von KI-Methoden zur Lösung von Problemen der Regelungstechnik}, year = 2023 }
• Organic Computing -- Doctoral Dissertation Colloquium 2022. Tomforde, Sven; Krupitzer, Christian. In Vol. 24Intelligent Embedded Systems. kassel university press, 2023.
  • [ BibTeX ]
  @book{tomforde2023organic, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {isac-www}, publisher = {kassel university press}, series = {Intelligent Embedded Systems}, title = {Organic Computing – Doctoral Dissertation Colloquium 2022}, volume = 24, year = 2023 }
• Integration and Evaluation of a High-Precision MotionCam-3D into a 3D Thermography System. Mendez, Miguel; Schramm, Sebastian; Schmoll, Robert; Kroll, Andreas. In Journal of Sensors and Sensor Systems (JSSS). 2023.
  • [ BibTeX ]
  • [ URL ]
  @article{Mendez2023c, author = {Mendez, Miguel and Schramm, Sebastian and Schmoll, Robert and Kroll, Andreas}, journal = {Journal of Sensors and Sensor Systems (JSSS)}, keywords = {isac-www}, title = {Integration and Evaluation of a High-Precision MotionCam-3D into a 3D Thermography System}, year = 2023 }
• Ausgewählte Beiträge aus dem GMA-Fachausschuss 5.14 "Computational Intelligence". Mikut, Ralf; Kroll, Andreas; Schulte, Horst. In at -- Automatisierungstechnik, 71(10), R. Mikut (red.), bll 817–819. 2023.
  • [ BibTeX ]
  @article{2023-Kroll-Automatisierungstechnik-CI, author = {Mikut, Ralf and Kroll, Andreas and Schulte, Horst}, editor = {Mikut, Ralf}, journal = {at – Automatisierungstechnik}, keywords = {isac-www}, note = {Editorial}, number = 10, pages = {817–819}, title = {Ausgewählte Beiträge aus dem GMA-Fachausschuss 5.14 "Computational Intelligence"}, volume = 71, year = 2023 }
• Leveraging Ensemble Structures to Elucidate the Impact of Factors that Influence the Quality of Ultra–High Performance Concrete. Rezazadeh, Farzad; Dürrbaum, Axel; Zimmermann, Gregor; Kroll, Andreas. In 2023 IEEE Symposium Series on Computational Intelligence (SSCI), bll 180–187. Mexico City, Mexico, 2023.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{RezazadehSSCI2023, address = {Mexico City, Mexico}, author = {Rezazadeh, Farzad and Dürrbaum, Axel and Zimmermann, Gregor and Kroll, Andreas}, booktitle = {2023 IEEE Symposium Series on Computational Intelligence (SSCI)}, keywords = {isac-www}, month = {5.-8. Dezember}, pages = {180-187}, title = {Leveraging Ensemble Structures to Elucidate the Impact of Factors that Influence the Quality of Ultra–High Performance Concrete}, year = 2023 }
• Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering. Aßenmacher, Matthias; Rauch, Lukas; Goschenhofer, Jann; Stephan, Andreas; Bischl, Bernd; Roth, Benjamin; Sick, Bernhard. In Workshop on Interactive Adapative Learning (IAL), ECML PKDD, bll 65–73. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.

  @inproceedings{assenmacher2023towards, abstract = {Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.}, author = {Aßenmacher, Matthias and Rauch, Lukas and Goschenhofer, Jann and Stephan, Andreas and Bischl, Bernd and Roth, Benjamin and Sick, Bernhard}, booktitle = {Workshop on Interactive Adapative Learning (IAL), ECML PKDD}, keywords = {isac-www}, pages = {65–73}, title = {Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering}, year = 2023 }
• Model Predictive Control of Wind Turbines with Piecewise-Affine Power Coefficient Approximation. Sterle, Arnold; Grapentin, Aaron; Hans, Christian A.; Raisch, Jörg. In 2023 62nd IEEE Conference on Decision and Control (CDC), bll 3586–3591. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  In this paper, an offset-free bilinear model predictive control approach for wind turbines is presented. State-of-the-art controllers employ different control loops for pitch angle and generator torque which switch depending on wind conditions. In contrast, the presented controller is based on one unified control law that works for all wind conditions. The inherent nonlinearity of wind turbines is addressed through a piecewise-affine approximation, which is modelled in a mixed-integer fashion. The presented controller is compared to a state-of-the-art baseline controller in a numerical case study using OpenFAST. Simulation results show that the presented controller ensures accurate reference power tracking. Additionally, damage equivalent loads are reduced for higher wind speeds.

  @inproceedings{10383567, abstract = {In this paper, an offset-free bilinear model predictive control approach for wind turbines is presented. State-of-the-art controllers employ different control loops for pitch angle and generator torque which switch depending on wind conditions. In contrast, the presented controller is based on one unified control law that works for all wind conditions. The inherent nonlinearity of wind turbines is addressed through a piecewise-affine approximation, which is modelled in a mixed-integer fashion. The presented controller is compared to a state-of-the-art baseline controller in a numerical case study using OpenFAST. Simulation results show that the presented controller ensures accurate reference power tracking. Additionally, damage equivalent loads are reduced for higher wind speeds.}, author = {Sterle, Arnold and Grapentin, Aaron and Hans, Christian A. and Raisch, Jörg}, booktitle = {2023 62nd IEEE Conference on Decision and Control (CDC)}, keywords = {isac-www}, month = {dec}, pages = {3586-3591}, title = {Model Predictive Control of Wind Turbines with Piecewise-Affine Power Coefficient Approximation}, year = 2023 }
• LQ Optimal Control for Power Tracking Operation of Wind Turbines. Grapentin, Aaron; Sterle, Arnold; Raisch, Jörg; Hans, Christian A. In IFAC-PapersOnLine, 56(2), bll 2759–2766. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper, an approach for active power control of individual wind turbines is presented. State-of-the-art controllers typically employ separate control loops for torque and pitch control. In contrast, we use a multivariable control approach. In detail, active power control is achieved by using reference trajectories for generator speed, generator torque, and pitch angle such that a desired power demand is met if weather conditions allow. Then, a linear quadratic (LQ) optimal controller is used for reference tracking. In an OpenFAST simulation environment, the controller is compared to a state-of-the-art approach. The simulations show a similar active power tracking performance, while the LQ optimal controller results in lower mechanical wear. Moreover, the presented approach exhibits good reference tracking and by improving the reference trajectory generation further performance increases can be expected.

  @article{GRAPENTIN20232759, abstract = {In this paper, an approach for active power control of individual wind turbines is presented. State-of-the-art controllers typically employ separate control loops for torque and pitch control. In contrast, we use a multivariable control approach. In detail, active power control is achieved by using reference trajectories for generator speed, generator torque, and pitch angle such that a desired power demand is met if weather conditions allow. Then, a linear quadratic (LQ) optimal controller is used for reference tracking. In an OpenFAST simulation environment, the controller is compared to a state-of-the-art approach. The simulations show a similar active power tracking performance, while the LQ optimal controller results in lower mechanical wear. Moreover, the presented approach exhibits good reference tracking and by improving the reference trajectory generation further performance increases can be expected.}, author = {Grapentin, Aaron and Sterle, Arnold and Raisch, Jörg and Hans, Christian A.}, journal = {IFAC-PapersOnLine}, keywords = {isac-www}, note = {22nd IFAC World Congress}, number = 2, pages = {2759-2766}, title = {LQ Optimal Control for Power Tracking Operation of Wind Turbines}, volume = 56, year = 2023 }
• Spatio-Temporal Attention Graph Neural Network for Remaining Useful Life Prediction. Huang, Zhixin; He, Yujiang; Sick, Bernhard. In Computational Science and Computational Intelligence (CSCI), bll 99–105. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  RUL prediction plays a crucial role in the health management of industrial systems. Given the increasing complexity of systems, data-driven predictive models have attracted significant research interest. Upon reviewing the existing literature, it appears that many studies either do not fully integrate both spatial and temporal features or employ only a single attention mechanism. Furthermore, there seems to be inconsistency in the choice of data normalization methods, particularly concerning operating conditions, which might influence predictive performance. To bridge these observations, this study presents the Spatio-Temporal Attention Graph Neural Network. Our model combines graph neural networks and temporal convolutional neural networks for spatial and temporal feature extraction, respectively. The cascade of these extractors, combined with multihead attention mechanisms for both spatio-temporal dimensions, aims to improve predictive precision and refine model explainability. Comprehensive experiments were conducted on the CMAPSS dataset to evaluate the impact of unified versus clustering normalization. The findings suggest that our model performs state-of-the-art results using only the unified normalization. Additionally, when dealing with datasets with multiple operating conditions, cluster normalization enhances the performance of our proposed model by up to 27%.

  @inproceedings{huang2023spatio, abstract = {RUL prediction plays a crucial role in the health management of industrial systems. Given the increasing complexity of systems, data-driven predictive models have attracted significant research interest. Upon reviewing the existing literature, it appears that many studies either do not fully integrate both spatial and temporal features or employ only a single attention mechanism. Furthermore, there seems to be inconsistency in the choice of data normalization methods, particularly concerning operating conditions, which might influence predictive performance. To bridge these observations, this study presents the Spatio-Temporal Attention Graph Neural Network. Our model combines graph neural networks and temporal convolutional neural networks for spatial and temporal feature extraction, respectively. The cascade of these extractors, combined with multihead attention mechanisms for both spatio-temporal dimensions, aims to improve predictive precision and refine model explainability. Comprehensive experiments were conducted on the CMAPSS dataset to evaluate the impact of unified versus clustering normalization. The findings suggest that our model performs state-of-the-art results using only the unified normalization. Additionally, when dealing with datasets with multiple operating conditions, cluster normalization enhances the performance of our proposed model by up to 27%.}, author = {Huang, Zhixin and He, Yujiang and Sick, Bernhard}, booktitle = {Computational Science and Computational Intelligence (CSCI)}, keywords = {Clustering_Normalization}, pages = {99–105}, publisher = {IEEE}, title = {Spatio-Temporal Attention Graph Neural Network for Remaining Useful Life Prediction}, year = 2023 }
• An Examination of Organic Computing Strategies in Design Optimization. Decke, Jens. In Organic Computing -- Doctoral Dissertation Colloquium 2023, S. Tomforde, C. Krupitzer (reds.), bll 41–54. kassel university press, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  In the field of design optimization, numerical equation solvers, commonly used for finite element analysis and computational fluid dy- namics, impose significant computational demands. The iterative nature of design optimization, involving numerous simulations, magnifies re- source requirements in terms of time, energy, costs, and greenhouse gas emissions. Accelerating this process has been a long-standing research challenge, driven by the potential for resource savings and the ability to tackle increasingly complex problems. Recently, organic computing methods have gained prominence as promising approaches to address this challenge. This study aims to bridge the gap by integrating tech- niques from deep learning, active learning, and generative learning into the field of design optimization. The goal is to accelerate the design op- timization process while addressing specific challenges such as exploring large and complex design spaces and evaluating the advantages and con- straints of these methods. This research has the potential to significantly impact the industrial use of design optimization by providing faster and more efficient tools.

  @incollection{decke2023examination, abstract = {In the field of design optimization, numerical equation solvers, commonly used for finite element analysis and computational fluid dy- namics, impose significant computational demands. The iterative nature of design optimization, involving numerous simulations, magnifies re- source requirements in terms of time, energy, costs, and greenhouse gas emissions. Accelerating this process has been a long-standing research challenge, driven by the potential for resource savings and the ability to tackle increasingly complex problems. Recently, organic computing methods have gained prominence as promising approaches to address this challenge. This study aims to bridge the gap by integrating tech- niques from deep learning, active learning, and generative learning into the field of design optimization. The goal is to accelerate the design op- timization process while addressing specific challenges such as exploring large and complex design spaces and evaluating the advantages and con- straints of these methods. This research has the potential to significantly impact the industrial use of design optimization by providing faster and more efficient tools.}, author = {Decke, Jens}, booktitle = {Organic Computing – Doctoral Dissertation Colloquium 2023}, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {GenerativeLearning}, pages = {41–54}, publisher = {kassel university press}, title = {An Examination of Organic Computing Strategies in Design Optimization}, year = 2023 }
• Active Label Refinement for Semantic Segmentation of Satellite Images. Pham, Minh Tuan; Wijesingha, Jayan; Kottke, Daniel; Herde, Marek; Huseljic, Denis; Sick, Bernhard; Wachendorf, Michael; Esch, Thomas. In arXiv e-prints, bl arXiv:2309.06159. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Remote sensing through semantic segmentation of satellite images contributes to the understanding and utilisation of the earth's surface. For this purpose, semantic segmentation networks are typically trained on large sets of labelled satellite images. However, obtaining expert labels for these images is costly. Therefore, we propose to rely on a low-cost approach, e.g. crowdsourcing or pretrained networks, to label the images in the first step. Since these initial labels are partially erroneous, we use active learning strategies to cost-efficiently refine the labels in the second step. We evaluate the active learning strategies using satellite images of Bengaluru in India, labelled with land cover and land use labels. Our experimental results suggest that an active label refinement to improve the semantic segmentation network's performance is beneficial.

  @article{pham2023active, abstract = {Remote sensing through semantic segmentation of satellite images contributes to the understanding and utilisation of the earth's surface. For this purpose, semantic segmentation networks are typically trained on large sets of labelled satellite images. However, obtaining expert labels for these images is costly. Therefore, we propose to rely on a low-cost approach, e.g. crowdsourcing or pretrained networks, to label the images in the first step. Since these initial labels are partially erroneous, we use active learning strategies to cost-efficiently refine the labels in the second step. We evaluate the active learning strategies using satellite images of Bengaluru in India, labelled with land cover and land use labels. Our experimental results suggest that an active label refinement to improve the semantic segmentation network's performance is beneficial.}, author = {Pham, Minh Tuan and Wijesingha, Jayan and Kottke, Daniel and Herde, Marek and Huseljic, Denis and Sick, Bernhard and Wachendorf, Michael and Esch, Thomas}, journal = {arXiv e-prints}, keywords = {isac-www}, pages = {arXiv:2309.06159}, title = {Active Label Refinement for Semantic Segmentation of Satellite Images}, year = 2023 }
• What Does Really Count? Estimating Relevance of Corner Cases for Semantic Segmentation in Automated Driving. Breitenstein, Jasmin; Heidecker, Florian; Lyssenko, Maria; Bogdoll, Daniel; Bieshaar, Maarten; Zöllner, J. Marius; Sick, Bernhard; Fingscheidt, Tim. In Workshop on roBustness and Reliability of Autonomous Vehicles in the Open-world (BRAVO), ICCV, bll 3991–4000. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In safety-critical applications such as automated driving, perception errors may create an imminent risk to vulnerable road users (VRU). To mitigate the occurrence of unexpected and potentially dangerous situations, so-called corner cases, perception models are trained on a huge amount of data. However, the models are typically evaluated using task-agnostic metrics, which do not reflect the severity of safety-critical misdetections. Consequently, misdetections with particular relevance for the safe driving task should entail a more severe penalty during evaluation to pinpoint corner cases in large-scale datasets. In this work, we propose a novel metric IoUw that exploits relevance on the pixel level of the semantic segmentation output to extend the notion of the intersection over union (IoU) by emphasizing small areas of an image affected by corner cases. We (i) employ IoUw to measure the effect of pre-defined relevance criteria on the segmentation evaluation, and (ii) use the relevance-adapted IoUw to refine the identification of corner cases. In our experiments, we investigate vision-based relevance criteria and physical attributes as per-pixel criticality to factor in the imminent risk, showing that IoUw precisely accentuates the relevance of corner cases.

  @inproceedings{breitenstein2023what, abstract = {In safety-critical applications such as automated driving, perception errors may create an imminent risk to vulnerable road users (VRU). To mitigate the occurrence of unexpected and potentially dangerous situations, so-called corner cases, perception models are trained on a huge amount of data. However, the models are typically evaluated using task-agnostic metrics, which do not reflect the severity of safety-critical misdetections. Consequently, misdetections with particular relevance for the safe driving task should entail a more severe penalty during evaluation to pinpoint corner cases in large-scale datasets. In this work, we propose a novel metric IoUw that exploits relevance on the pixel level of the semantic segmentation output to extend the notion of the intersection over union (IoU) by emphasizing small areas of an image affected by corner cases. We (i) employ IoUw to measure the effect of pre-defined relevance criteria on the segmentation evaluation, and (ii) use the relevance-adapted IoUw to refine the identification of corner cases. In our experiments, we investigate vision-based relevance criteria and physical attributes as per-pixel criticality to factor in the imminent risk, showing that IoUw precisely accentuates the relevance of corner cases.}, author = {Breitenstein, Jasmin and Heidecker, Florian and Lyssenko, Maria and Bogdoll, Daniel and Bieshaar, Maarten and Zöllner, J. Marius and Sick, Bernhard and Fingscheidt, Tim}, booktitle = {Workshop on roBustness and Reliability of Autonomous Vehicles in the Open-world (BRAVO), ICCV}, keywords = {isac-www}, pages = {3991–4000}, title = {What Does Really Count? Estimating Relevance of Corner Cases for Semantic Segmentation in Automated Driving}, year = 2023 }
• Model selection, adaptation, and combination for transfer learning in wind and photovoltaic power forecasts. Schreiber, Jens; Sick, Bernhard. In Energy and AI, 14, bl 100249. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  There is recent interest in using model hubs – a collection of pre-trained models – in computer vision tasks. To employ a model hub, we first select a source model and then adapt the model for the target to compensate for differences. There still needs to be more research on model selection and adaption for renewable power forecasts. In particular, none of the related work examines different model selection and adaptation strategies for neural network architectures. Also, none of the current studies investigates the influence of available training samples and considers seasonality in the evaluation. We close these gaps by conducting the first thorough experiment for model selection and adaptation for transfer learning in renewable power forecast, adopting recent developments from the field of computer vision on 667 wind and photovoltaic parks from six datasets. We simulate different amounts of training samples for each season to calculate informative forecast errors. We examine the marginal likelihood and forecast error for model selection for those amounts. Furthermore, we study four adaption strategies. As an extension of the current state of the art, we utilize a Bayesian linear regression for forecasting the response based on features extracted from a neural network. This approach outperforms the baseline with only seven days of training data and shows that fine-tuning is not beneficial with less than three months of data. We further show how combining multiple models through ensembles can significantly improve the model selection and adaptation approach such that we have a similar mean error with only 30 days of training data which is otherwise only possible with an entire year of training data. We achieve a mean error of 9.8 and 14 percent for the most realistic dataset for PV and wind with only seven days of training data.

  @article{schreiber2023model, abstract = {There is recent interest in using model hubs – a collection of pre-trained models – in computer vision tasks. To employ a model hub, we first select a source model and then adapt the model for the target to compensate for differences. There still needs to be more research on model selection and adaption for renewable power forecasts. In particular, none of the related work examines different model selection and adaptation strategies for neural network architectures. Also, none of the current studies investigates the influence of available training samples and considers seasonality in the evaluation. We close these gaps by conducting the first thorough experiment for model selection and adaptation for transfer learning in renewable power forecast, adopting recent developments from the field of computer vision on 667 wind and photovoltaic parks from six datasets. We simulate different amounts of training samples for each season to calculate informative forecast errors. We examine the marginal likelihood and forecast error for model selection for those amounts. Furthermore, we study four adaption strategies. As an extension of the current state of the art, we utilize a Bayesian linear regression for forecasting the response based on features extracted from a neural network. This approach outperforms the baseline with only seven days of training data and shows that fine-tuning is not beneficial with less than three months of data. We further show how combining multiple models through ensembles can significantly improve the model selection and adaptation approach such that we have a similar mean error with only 30 days of training data which is otherwise only possible with an entire year of training data. We achieve a mean error of 9.8 and 14 percent for the most realistic dataset for PV and wind with only seven days of training data.}, author = {Schreiber, Jens and Sick, Bernhard}, journal = {Energy and AI}, keywords = {isac-www}, pages = 100249, title = {Model selection, adaptation, and combination for transfer learning in wind and photovoltaic power forecasts}, volume = 14, year = 2023 }
• It is the habit not the handle that affects tooth brushing. Results of a randomised counterbalanced cross over study. Deinzer, Renate; Eidenhardt, Zdenka; Sohrabi, Keywan; Stenger, Manuel; Kraft, Dominik; Sick, Bernhard; Götz-Hahn, Franz; Bottenbruch, Carlotta; Berneburg, Nils; Weik, Ulrike. In Research Square. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Objective: To assess the effect of the toothbrush handle on video-observed toothbrushing behaviour and toothbrushing effectiveness Methods: This is a randomized counterbalanced cross-over study. N = 50 university students brushed their teeth at two occasions, one week apart, using either a commercial ergonomically designed manual toothbrush (MT) or Brushalyze V1 (BV1), a manual toothbrush with a thick cylindrical handle without any specific ergonomic features. Brushing behaviour was video-analysed. Plaque was assessed at the second occasion immediately after brushing. Participants directly compared the two brushes regarding their handling and compared them to the brushed they used at home. Results: The study participants found the BV1 significantly more cumbersome than the M1 or their brush at home. (p < 0.05). However, correlation analyses revealed a strong consistency of brushing behavior with the two brushes (0.71 < r < 0.91). Means differed only slightly (all d < 0.36). These differences became statistically significant only for the brushing time at inner surfaces (d = 0.31 p = 0.03) and horizontal movements at inner surfaces (d = 0.35, p = 0.02). Plaque levels at the gingival margins did not differ while slightly more plaque persisted at the more coronal aspects of the crown after brushing with BV1 (d = 0.592; p 0.042) Discussion: The results of the study indicate that the brushing handle does not play a major role in brushing behavior or brushing effectiveness.

  @article{deinzer2023it, abstract = {Objective: To assess the effect of the toothbrush handle on video-observed toothbrushing behaviour and toothbrushing effectiveness Methods: This is a randomized counterbalanced cross-over study. N = 50 university students brushed their teeth at two occasions, one week apart, using either a commercial ergonomically designed manual toothbrush (MT) or Brushalyze V1 (BV1), a manual toothbrush with a thick cylindrical handle without any specific ergonomic features. Brushing behaviour was video-analysed. Plaque was assessed at the second occasion immediately after brushing. Participants directly compared the two brushes regarding their handling and compared them to the brushed they used at home. Results: The study participants found the BV1 significantly more cumbersome than the M1 or their brush at home. (p < 0.05). However, correlation analyses revealed a strong consistency of brushing behavior with the two brushes (0.71 < r < 0.91). Means differed only slightly (all d < 0.36). These differences became statistically significant only for the brushing time at inner surfaces (d = 0.31 p = 0.03) and horizontal movements at inner surfaces (d = 0.35, p = 0.02). Plaque levels at the gingival margins did not differ while slightly more plaque persisted at the more coronal aspects of the crown after brushing with BV1 (d = 0.592; p 0.042) Discussion: The results of the study indicate that the brushing handle does not play a major role in brushing behavior or brushing effectiveness.}, author = {Deinzer, Renate and Eidenhardt, Zdenka and Sohrabi, Keywan and Stenger, Manuel and Kraft, Dominik and Sick, Bernhard and Götz-Hahn, Franz and Bottenbruch, Carlotta and Berneburg, Nils and Weik, Ulrike}, journal = {Research Square}, keywords = {isac-www}, note = {(preprint)}, title = {It is the habit not the handle that affects tooth brushing. Results of a randomised counterbalanced cross over study}, year = 2023 }
• Corner Cases in Machine Learning Processes. Heidecker, Florian; Bieshaar, Maarten; Sick, Bernhard. In AI Perspectives & Advances, 6(1), bll 1–17. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Applications using machine learning (ML), such as highly autonomous driving, depend highly on the performance of the ML model. The data amount and quality used for model training and validation are crucial. If the model cannot detect and interpret a new, rare, or perhaps dangerous situation, often referred to as a corner case, we will likely blame the data for not being good enough or too small in number. However, the implemented ML model and its associated architecture also influence the behavior. Therefore, the occurrence of prediction errors resulting from the ML model itself is not surprising. This work addresses a corner case definition from an ML model's perspective to determine which aspects must be considered. To achieve this goal, we present an overview of properties for corner cases that are beneficial for the description, explanation, reproduction, or synthetic generation of corner cases. To define ML corner cases, we review different considerations in the literature and summarize them in a general description and mathematical formulation, whereby the expected relevance-weighted loss is the key to distinguishing corner cases from common data. Moreover, we show how to operationalize the corner case characteristics to determine the value of a corner case. To conclude, we present the extended taxonomy for ML corner cases by adding the input, model, and deployment levels, considering the influence of the corner case properties.

  @article{heidecker2023corner, abstract = {Applications using machine learning (ML), such as highly autonomous driving, depend highly on the performance of the ML model. The data amount and quality used for model training and validation are crucial. If the model cannot detect and interpret a new, rare, or perhaps dangerous situation, often referred to as a corner case, we will likely blame the data for not being good enough or too small in number. However, the implemented ML model and its associated architecture also influence the behavior. Therefore, the occurrence of prediction errors resulting from the ML model itself is not surprising. This work addresses a corner case definition from an ML model's perspective to determine which aspects must be considered. To achieve this goal, we present an overview of properties for corner cases that are beneficial for the description, explanation, reproduction, or synthetic generation of corner cases. To define ML corner cases, we review different considerations in the literature and summarize them in a general description and mathematical formulation, whereby the expected relevance-weighted loss is the key to distinguishing corner cases from common data. Moreover, we show how to operationalize the corner case characteristics to determine the value of a corner case. To conclude, we present the extended taxonomy for ML corner cases by adding the input, model, and deployment levels, considering the influence of the corner case properties.}, author = {Heidecker, Florian and Bieshaar, Maarten and Sick, Bernhard}, journal = {AI Perspectives & Advances}, keywords = {Taxonomy}, number = 1, pages = {1–17}, title = {Corner Cases in Machine Learning Processes}, volume = 6, year = 2023 }
• Continuous Feature Networks: A Novel Method to Process Irregularly and Inconsistently Sampled Data With Position-Dependent Features. Magnussen, Birk Martin; Stern, Claudius; Sick, Bernhard. In International Journal On Advances in Intelligent Systems, 16(3&4), bll 43–50. ThinkMind, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Continuous Kernels have been a recent development in convolutional neural networks. Such kernels are used to process data sampled at different resolutions as well as irregularly and inconsistently sampled data. Convolutional neural networks have the property of translational invariance (e.g., features are detected regardless of their position in the measurement domain), which is unsuitable if the position of detected features is relevant for the prediction task. However, the capabilities of continuous kernels to process irregularly sampled data are still desired. This article introduces the continuous feature network, a novel method utilizing continuous kernels, for detecting global features at absolute positions in the data domain. Through a use case in processing multiple spatially resolved reflection spectroscopy data, which is sampled irregularly and inconsistently, we show that the proposed method is capable of processing such data directly without additional preprocessing or augmentation as is needed using comparable methods. In addition, we show that the proposed method is able to achieve a higher prediction accuracy than a comparable network on a dataset with position-dependent features. Furthermore, a higher robustness to missing data compared to a benchmark network using data interpolation is observed, which allows the network to adapt to sensors with a failure of individual light emitters or detectors without the need for retraining. The article shows how these capabilities stem from the continuous kernels used and how the number of available kernels to be trained affects the model. Finally, the article proposes a method to utilize the introduced method as a base for an interpretable model usable for explainable AI.

  @article{magnussen2023continuous, abstract = {Continuous Kernels have been a recent development in convolutional neural networks. Such kernels are used to process data sampled at different resolutions as well as irregularly and inconsistently sampled data. Convolutional neural networks have the property of translational invariance (e.g., features are detected regardless of their position in the measurement domain), which is unsuitable if the position of detected features is relevant for the prediction task. However, the capabilities of continuous kernels to process irregularly sampled data are still desired. This article introduces the continuous feature network, a novel method utilizing continuous kernels, for detecting global features at absolute positions in the data domain. Through a use case in processing multiple spatially resolved reflection spectroscopy data, which is sampled irregularly and inconsistently, we show that the proposed method is capable of processing such data directly without additional preprocessing or augmentation as is needed using comparable methods. In addition, we show that the proposed method is able to achieve a higher prediction accuracy than a comparable network on a dataset with position-dependent features. Furthermore, a higher robustness to missing data compared to a benchmark network using data interpolation is observed, which allows the network to adapt to sensors with a failure of individual light emitters or detectors without the need for retraining. The article shows how these capabilities stem from the continuous kernels used and how the number of available kernels to be trained affects the model. Finally, the article proposes a method to utilize the introduced method as a base for an interpretable model usable for explainable AI.}, author = {Magnussen, Birk Martin and Stern, Claudius and Sick, Bernhard}, journal = {International Journal On Advances in Intelligent Systems}, keywords = {reflection_spectroscopy}, number = {3&4}, pages = {43–50}, publisher = {ThinkMind}, title = {Continuous Feature Networks: A Novel Method to Process Irregularly and Inconsistently Sampled Data With Position-Dependent Features}, volume = 16, year = 2023 }
• Targeted Adversarial Attacks on Wind Power Forecasts. Heinrich, René; Scholz, Christoph; Vogt, Stephan; Lehna, Malte. In Machine Learning, 113(2), bll 863–889. Springer, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of individual wind farms and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.

  @article{heinrich2023targeted, abstract = {In recent years, researchers proposed a variety of deep learning models for wind power forecasting. These models predict the wind power generation of wind farms or entire regions more accurately than traditional machine learning algorithms or physical models. However, latest research has shown that deep learning models can often be manipulated by adversarial attacks. Since wind power forecasts are essential for the stability of modern power systems, it is important to protect them from this threat. In this work, we investigate the vulnerability of two different forecasting models to targeted, semi-targeted, and untargeted adversarial attacks. We consider a Long Short-Term Memory (LSTM) network for predicting the power generation of individual wind farms and a Convolutional Neural Network (CNN) for forecasting the wind power generation throughout Germany. Moreover, we propose the Total Adversarial Robustness Score (TARS), an evaluation metric for quantifying the robustness of regression models to targeted and semi-targeted adversarial attacks. It assesses the impact of attacks on the model's performance, as well as the extent to which the attacker's goal was achieved, by assigning a score between 0 (very vulnerable) and 1 (very robust). In our experiments, the LSTM forecasting model was fairly robust and achieved a TARS value of over 0.78 for all adversarial attacks investigated. The CNN forecasting model only achieved TARS values below 0.10 when trained ordinarily, and was thus very vulnerable. Yet, its robustness could be significantly improved by adversarial training, which always resulted in a TARS above 0.46.}, author = {Heinrich, René and Scholz, Christoph and Vogt, Stephan and Lehna, Malte}, journal = {Machine Learning}, keywords = {isac-www}, number = 2, pages = {863–889}, publisher = {Springer}, title = {Targeted Adversarial Attacks on Wind Power Forecasts}, volume = 113, year = 2023 }
• Time-aware Robustness of Temporal Graph Neural Networks for Link Prediction. Sälzer, Marco; Beddar-Wiesing, Silvia. In International Symposium on Temporal Representation and Reasoning (TIME), bll 19:1–19:3. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Graph Neural Networks (GNNs) provide a framework for computing functions over graphs based on learnable parameters, which gained much attention in recent years. The most popular GNN models, so-called convolutional GNN or message-passing GNN apply a neighborhood aggregation procedure to each node in a graph to compute its output. Usually, such GNNs are used for classification or prediction tasks over static graphs. However, this limits their applicability in contexts like social networks or knowledge graphs, where underlying graphs change stepwise or time-continuously. Temporal Graph Neural Networks (TGNN) try to close this gap. The general idea of TGNN is to generalize the neighborhood aggregation procedure mentioned above to temporal graphs, usually represented as a tuple of a base graph with a series of time-stamped observed changes. In most applications involving Neural Network based models, giving reliable safety certificates are highly desirable but also a significant challenge, especially because of the blackbox nature of neural models. In this extended abstract, we address the topic of verifying TGNN, which is an unexplored area of research. We present a first notion of a time-aware robustness property for TGNN used for link prediction tasks, motivated by recent work on similar time-aware attacks. Furthermore, we discuss our ongoing work regarding promising verification approaches for the presented or similar safety properties and possible next steps in this research direction.

  @inproceedings{salzer2023time, abstract = {Graph Neural Networks (GNNs) provide a framework for computing functions over graphs based on learnable parameters, which gained much attention in recent years. The most popular GNN models, so-called convolutional GNN or message-passing GNN apply a neighborhood aggregation procedure to each node in a graph to compute its output. Usually, such GNNs are used for classification or prediction tasks over static graphs. However, this limits their applicability in contexts like social networks or knowledge graphs, where underlying graphs change stepwise or time-continuously. Temporal Graph Neural Networks (TGNN) try to close this gap. The general idea of TGNN is to generalize the neighborhood aggregation procedure mentioned above to temporal graphs, usually represented as a tuple of a base graph with a series of time-stamped observed changes. In most applications involving Neural Network based models, giving reliable safety certificates are highly desirable but also a significant challenge, especially because of the blackbox nature of neural models. In this extended abstract, we address the topic of verifying TGNN, which is an unexplored area of research. We present a first notion of a time-aware robustness property for TGNN used for link prediction tasks, motivated by recent work on similar time-aware attacks. Furthermore, we discuss our ongoing work regarding promising verification approaches for the presented or similar safety properties and possible next steps in this research direction.}, author = {Sälzer, Marco and Beddar-Wiesing, Silvia}, booktitle = {International Symposium on Temporal Representation and Reasoning (TIME)}, keywords = {isac-www}, pages = {19:1–19:3}, title = {Time-aware Robustness of Temporal Graph Neural Networks for Link Prediction}, year = 2023 }
• Height Change Feature Based Free Space Detection. Schreck, Steven; Reichert, Hannes; Hetzel, Manuel; Doll, Konrad; Sick, Bernhard. In International Conference on Control, Mechatronics and Automation (ICCMA), bll 171–176. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  We present a novel method for free space detection in dynamic environments like factory sites, crucial for autonomous forklifts to avoid collisions. It introduces a technique for fast surface normal estimation using spherical projected LiDAR data, which helps in detecting free space efficiently and in real-time. The method's effectiveness is proven with a 50.90% mIoU score on the Semantic KITTI dataset at 105 Hz, and a 63.30% mIoU score on a factory site dataset at 54 Hz.

  @inproceedings{schreck2023height, abstract = {We present a novel method for free space detection in dynamic environments like factory sites, crucial for autonomous forklifts to avoid collisions. It introduces a technique for fast surface normal estimation using spherical projected LiDAR data, which helps in detecting free space efficiently and in real-time. The method's effectiveness is proven with a 50.90% mIoU score on the Semantic KITTI dataset at 105 Hz, and a 63.30% mIoU score on a factory site dataset at 54 Hz.}, author = {Schreck, Steven and Reichert, Hannes and Hetzel, Manuel and Doll, Konrad and Sick, Bernhard}, booktitle = {International Conference on Control, Mechatronics and Automation (ICCMA)}, keywords = {isac-www}, pages = {171–176}, publisher = {IEEE}, title = {Height Change Feature Based Free Space Detection}, year = 2023 }
• Machine Learning Supported Optimisation and Experimental Evaluation of Electrical Motors for Small Urban Passenger Vehicles. Botache, Diego. In Organic Computing -- Doctoral Dissertation Colloquium 2023, S. Tomforde, C. Krupitzer (reds.), bll 14–26. kassel university press, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  The increasing environmental pollution and rising global energy demands require the rapid development of novel and sustainable traction machines from urban transport up to large-scale transport vehicles. The optimisation of the development cycle of electrical traction machines plays a vital role in sustainable mobility. Still, developers have to face the difficulty of analysing time-consuming multiphysics simulations to assess the performance of many prototype variants. Accelerating this process involves implementing machine learning-supported strategies that can effectively acquire results and predictions about the performance and efficiency of prototypes in an iterative optimisation process and detect possible issues during the experimental procedure. The following research project streamlines the development cycle by identifying and presenting multiple strategies for applying and evaluating machine learning and deep learning techniques in different stages and areas in the development cycle including self-optimisation and self-adaptation capabilities. By implementing these strategies, the optimisation process can be further improved and become more efficient and robust, reducing the time and cost required for testing and validation. Therefore, this proposal addresses two application areas. The first area consists of the supported machine learning motor performance prediction at the topology design and optimisation stage of structural components. The second area corresponds to the machine learning-supported experimental evaluation process of prototypes at desired test benches.

  @incollection{botache2023machine, abstract = {The increasing environmental pollution and rising global energy demands require the rapid development of novel and sustainable traction machines from urban transport up to large-scale transport vehicles. The optimisation of the development cycle of electrical traction machines plays a vital role in sustainable mobility. Still, developers have to face the difficulty of analysing time-consuming multiphysics simulations to assess the performance of many prototype variants. Accelerating this process involves implementing machine learning-supported strategies that can effectively acquire results and predictions about the performance and efficiency of prototypes in an iterative optimisation process and detect possible issues during the experimental procedure. The following research project streamlines the development cycle by identifying and presenting multiple strategies for applying and evaluating machine learning and deep learning techniques in different stages and areas in the development cycle including self-optimisation and self-adaptation capabilities. By implementing these strategies, the optimisation process can be further improved and become more efficient and robust, reducing the time and cost required for testing and validation. Therefore, this proposal addresses two application areas. The first area consists of the supported machine learning motor performance prediction at the topology design and optimisation stage of structural components. The second area corresponds to the machine learning-supported experimental evaluation process of prototypes at desired test benches.}, author = {Botache, Diego}, booktitle = {Organic Computing – Doctoral Dissertation Colloquium 2023}, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {test_benches}, pages = {14–26}, publisher = {kassel university press}, title = {Machine Learning Supported Optimisation and Experimental Evaluation of Electrical Motors for Small Urban Passenger Vehicles}, year = 2023 }
• Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions. Nivarthi, Chandana Priya; Vogt, Stephan; Sick, Bernhard. In Machine Learning and Knowledge Extraction (MAKE), 5(3), bll 1214–1233. MDPI, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Typically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from source parks to target parks. Nevertheless, determining the most similar source park(s) for transfer learning can be challenging, particularly when the target park has limited or no historical data samples. To address this issue, we propose a multi-task learning architecture that employs a Unified Autoencoder (UAE) to initially learn a common representation of input weather features among tasks and then utilizes a Task-Embedding layer in a Neural Network (TENN) to learn task-specific information. This proposed UAE-TENN architecture can be easily extended to new parks with or without historical data. We evaluate the performance of our proposed architecture and compare it to single-task learning models on six photovoltaic and wind farm datasets consisting of a total of 529 parks. Our results show that the UAE-TENN architecture significantly improves power-forecasting performance by 10 to 19% for photovoltaic parks and 5 to 15% for wind parks compared to baseline models. We also demonstrate that UAE-TENN improves forecast accuracy for a new park by 19% for photovoltaic parks, even in a zero-shot learning scenario where there is no historical data. Additionally, we propose variants of the Unified Autoencoder with convolutional and LSTM layers, compare their performance, and provide a comparison among architectures with different numbers of task-embedding dimensions. Finally, we demonstrate the utility of trained task embeddings for interpretation and visualization purposes.

  @article{nivarthi2023multi, abstract = {Typically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from source parks to target parks. Nevertheless, determining the most similar source park(s) for transfer learning can be challenging, particularly when the target park has limited or no historical data samples. To address this issue, we propose a multi-task learning architecture that employs a Unified Autoencoder (UAE) to initially learn a common representation of input weather features among tasks and then utilizes a Task-Embedding layer in a Neural Network (TENN) to learn task-specific information. This proposed UAE-TENN architecture can be easily extended to new parks with or without historical data. We evaluate the performance of our proposed architecture and compare it to single-task learning models on six photovoltaic and wind farm datasets consisting of a total of 529 parks. Our results show that the UAE-TENN architecture significantly improves power-forecasting performance by 10 to 19% for photovoltaic parks and 5 to 15% for wind parks compared to baseline models. We also demonstrate that UAE-TENN improves forecast accuracy for a new park by 19% for photovoltaic parks, even in a zero-shot learning scenario where there is no historical data. Additionally, we propose variants of the Unified Autoencoder with convolutional and LSTM layers, compare their performance, and provide a comparison among architectures with different numbers of task-embedding dimensions. Finally, we demonstrate the utility of trained task embeddings for interpretation and visualization purposes.}, author = {Nivarthi, Chandana Priya and Vogt, Stephan and Sick, Bernhard}, journal = {Machine Learning and Knowledge Extraction (MAKE)}, keywords = {isac-www}, number = 3, pages = {1214–1233}, publisher = {MDPI}, title = {Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions}, volume = 5, year = 2023 }
• Leveraging Repeated Unlabelled Noisy Measurements to Augment Supervised Learning. Magnussen, Birk Martin; Stern, Claudius; Sick, Bernhard. In International Conference on Computational Intelligence and Intelligent Systems (CIIS), bll 1–6. ACM, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Often, producing large labelled datasets for supervised machine learning is difficult and expensive. In cases where the expensive part is due to labelling and obtaining ground truth, it is often comparably easy to acquire large datasets containing unlabelled data points. For reproducible measurements, it is possible to record information on multiple data points being from the same reproducible measurement series, which should thus have an equal but unknown ground truth. In this article, we propose a method to incorporate a dataset of such unlabelled data points for which some data points are known to be equal in end-to-end training of otherwise labelled data. We show that, with the example of predicting the carotenoid concentration in human skin from optical multiple spatially resolved reflection spectroscopy data, the proposed method is capable of reducing the required number of labelled data points to achieve the same prediction accuracy for different model architectures. In addition, we show that the proposed method is capable of reducing the negative impact of noisy data when performing a repeated measurement of the same sample.

  @inproceedings{magnussen2023leveraging, abstract = {Often, producing large labelled datasets for supervised machine learning is difficult and expensive. In cases where the expensive part is due to labelling and obtaining ground truth, it is often comparably easy to acquire large datasets containing unlabelled data points. For reproducible measurements, it is possible to record information on multiple data points being from the same reproducible measurement series, which should thus have an equal but unknown ground truth. In this article, we propose a method to incorporate a dataset of such unlabelled data points for which some data points are known to be equal in end-to-end training of otherwise labelled data. We show that, with the example of predicting the carotenoid concentration in human skin from optical multiple spatially resolved reflection spectroscopy data, the proposed method is capable of reducing the required number of labelled data points to achieve the same prediction accuracy for different model architectures. In addition, we show that the proposed method is capable of reducing the negative impact of noisy data when performing a repeated measurement of the same sample.}, author = {Magnussen, Birk Martin and Stern, Claudius and Sick, Bernhard}, booktitle = {International Conference on Computational Intelligence and Intelligent Systems (CIIS)}, keywords = {reflection_spectroscopy}, pages = {1–6}, publisher = {ACM}, title = {Leveraging Repeated Unlabelled Noisy Measurements to Augment Supervised Learning}, year = 2023 }
• Graph Pooling Provably Improves Expressivity. Lachi, Veronica; Moallemy-Oureh, Alice; Roth, Andreas; Welke, Pascal. In Workshop on New Frontiers in Graph Learning, NeurIPS, bll 1–7. 2023.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In the domain of graph neural networks (GNNs), pooling operators are fundamental to reduce the size of the graph by simplifying graph structures and vertex features. Recent advances have shown that well-designed pooling operators, coupled with message-passing layers, can endow hierarchical GNNs with an expressive power regarding the graph isomorphism test that is equal to the Weisfeiler-Leman test. However, the ability of hierarchical GNNs to increase expressive power by utilizing graph coarsening was not yet explored. This results in uncertainties about the benefits of pooling operators and a lack of sufficient properties to guide their design. In this work, we identify conditions for pooling operators to generate WL-distinguishable coarsened graphs from originally WL-indistinguishable but non-isomorphic graphs. Our conditions are versatile and can be tailored to specific tasks and data characteristics, offering a promising avenue for further research.

  @inproceedings{lachi2023graph, abstract = {In the domain of graph neural networks (GNNs), pooling operators are fundamental to reduce the size of the graph by simplifying graph structures and vertex features. Recent advances have shown that well-designed pooling operators, coupled with message-passing layers, can endow hierarchical GNNs with an expressive power regarding the graph isomorphism test that is equal to the Weisfeiler-Leman test. However, the ability of hierarchical GNNs to increase expressive power by utilizing graph coarsening was not yet explored. This results in uncertainties about the benefits of pooling operators and a lack of sufficient properties to guide their design. In this work, we identify conditions for pooling operators to generate WL-distinguishable coarsened graphs from originally WL-indistinguishable but non-isomorphic graphs. Our conditions are versatile and can be tailored to specific tasks and data characteristics, offering a promising avenue for further research.}, author = {Lachi, Veronica and Moallemy-Oureh, Alice and Roth, Andreas and Welke, Pascal}, booktitle = {Workshop on New Frontiers in Graph Learning, NeurIPS}, keywords = {isac-www}, pages = {1–7}, title = {Graph Pooling Provably Improves Expressivity}, year = 2023 }
• Context Information for Corner Case Detection in Highly Automated Driving. Heidecker, Florian; Susetzky, Tobias; Fuchs, Erich; Sick, Bernhard. In IEEE International Conference on Intelligent Transportation Systems (ITSC), bll 1522–1529. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Context information provided along with a dataset can be very helpful for solving a problem because the additional knowledge is already available and does not need to be extracted. Moreover, the context indicates how diverse a dataset is, i.e., how many samples per context category are available to train and test machine learning (ML) models. In this article, we present context annotations for the BDD100k image dataset. The annotations comprise, for instance, information about daytime, road condition (dry/wet), and dirt on the windshield. Sometimes, no or only little data are available for unique or rare combinations of these context attributes. However, data that matches these context conditions is crucial when discussing corner cases: Firstly, most ML models, e.g., object detectors, are not trained on such data, which leads to the assumption that they will perform poorly in those situations. Secondly, data containing corner cases are required for validating ML models. With this in mind, separate ML models dedicated to context detection are useful for expanding the training set with additional data of special interest, such as corner cases.

  @inproceedings{heidecker2023context, abstract = {Context information provided along with a dataset can be very helpful for solving a problem because the additional knowledge is already available and does not need to be extracted. Moreover, the context indicates how diverse a dataset is, i.e., how many samples per context category are available to train and test machine learning (ML) models. In this article, we present context annotations for the BDD100k image dataset. The annotations comprise, for instance, information about daytime, road condition (dry/wet), and dirt on the windshield. Sometimes, no or only little data are available for unique or rare combinations of these context attributes. However, data that matches these context conditions is crucial when discussing corner cases: Firstly, most ML models, e.g., object detectors, are not trained on such data, which leads to the assumption that they will perform poorly in those situations. Secondly, data containing corner cases are required for validating ML models. With this in mind, separate ML models dedicated to context detection are useful for expanding the training set with additional data of special interest, such as corner cases.}, author = {Heidecker, Florian and Susetzky, Tobias and Fuchs, Erich and Sick, Bernhard}, booktitle = {IEEE International Conference on Intelligent Transportation Systems (ITSC)}, keywords = {isac-www}, pages = {1522–1529}, publisher = {IEEE}, title = {Context Information for Corner Case Detection in Highly Automated Driving}, year = 2023 }
• Towards Few-Shot Time Series Anomaly Detection with Temporal Attention and Dynamic Thresholding. Nivarthi, Chandana Priya; Sick, Bernhard. In International Conference on Machine Learning and Applications (ICMLA), bll 1444–1450. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Anomaly detection plays a pivotal role in diverse realworld applications such as cybersecurity, fault detection, network monitoring, predictive maintenance, and highly automated driving. However, obtaining labeled anomalous data can be a formidable challenge, especially when anomalies exhibit temporal evolution. This paper introduces LATAM (Long short-term memory Autoencoder with Temporal Attention Mechanism) for few-shot anomaly detection, with the aim of enhancing detection performance in scenarios with limited labeled anomaly data. LATAM effectively captures temporal dependencies and emphasizes significant patterns in multivariate time series data. In our investigation, we comprehensively evaluate LATAM against other anomaly detection models, particularly assessing its capability in few-shot learning scenarios where we have minimal examples from the normal class and none from the anomalous class in the training data. Our experimental results, derived from real-world photovoltaic inverter data, highlight LATAM’s superiority, showcasing a substantial 27% mean F1 score improvement, even when trained on a mere two-week dataset. Furthermore, LATAM demonstrates remarkable results on the open-source SWaT dataset, achieving a 12% boost in accuracy with only two days of training data. Moreover, we introduce a simple yet effective dynamic thresholding mechanism, further enhancing the anomaly detection capabilities of LATAM. This underscores LATAM’s efficacy in addressing the challenges posed by limited labeled anomalies in practical scenarios and it proves valuable for downstream tasks involving temporal representation and time series prediction, extending its utility beyond anomaly detection applications.

  @inproceedings{nivarthi2023towards, abstract = {Anomaly detection plays a pivotal role in diverse realworld applications such as cybersecurity, fault detection, network monitoring, predictive maintenance, and highly automated driving. However, obtaining labeled anomalous data can be a formidable challenge, especially when anomalies exhibit temporal evolution. This paper introduces LATAM (Long short-term memory Autoencoder with Temporal Attention Mechanism) for few-shot anomaly detection, with the aim of enhancing detection performance in scenarios with limited labeled anomaly data. LATAM effectively captures temporal dependencies and emphasizes significant patterns in multivariate time series data. In our investigation, we comprehensively evaluate LATAM against other anomaly detection models, particularly assessing its capability in few-shot learning scenarios where we have minimal examples from the normal class and none from the anomalous class in the training data. Our experimental results, derived from real-world photovoltaic inverter data, highlight LATAM’s superiority, showcasing a substantial 27% mean F1 score improvement, even when trained on a mere two-week dataset. Furthermore, LATAM demonstrates remarkable results on the open-source SWaT dataset, achieving a 12% boost in accuracy with only two days of training data. Moreover, we introduce a simple yet effective dynamic thresholding mechanism, further enhancing the anomaly detection capabilities of LATAM. This underscores LATAM’s efficacy in addressing the challenges posed by limited labeled anomalies in practical scenarios and it proves valuable for downstream tasks involving temporal representation and time series prediction, extending its utility beyond anomaly detection applications.}, author = {Nivarthi, Chandana Priya and Sick, Bernhard}, booktitle = {International Conference on Machine Learning and Applications (ICMLA)}, keywords = {thresholding}, pages = {1444–1450}, publisher = {IEEE}, title = {Towards Few-Shot Time Series Anomaly Detection with Temporal Attention and Dynamic Thresholding}, year = 2023 }
• DADO – Low-Cost Query Strategies for Deep Active Design Optimization. Decke, Jens; Gruhl, Christian; Rauch, Lukas; Sick, Bernhard. In International Conference on Machine Learning and Applications (ICMLA), bll 1611–1618. IEEE, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.

  @inproceedings{decke2023dado, abstract = {In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.}, author = {Decke, Jens and Gruhl, Christian and Rauch, Lukas and Sick, Bernhard}, booktitle = {International Conference on Machine Learning and Applications (ICMLA)}, keywords = {Numerical-Simulation}, pages = {1611–1618}, publisher = {IEEE}, title = {DADO – Low-Cost Query Strategies for Deep Active Design Optimization}, year = 2023 }
• Context-aware recommendations for extended electric vehicle battery lifetime. Eider, Markus; Sick, Bernhard; Berl, Andreas. In Sustainable Computing: Informatics and Systems (SUSCOM), 37, bl 100845. Elsevier, 2023.
  • [ Abstract ]
  • [ BibTeX ]
  Electric vehicles are a means of reducing CO2 emissions in transportation. However, the sustainability of electric vehicle batteries is affected by battery health degradation, which decreases their overall lifetime. This results in a substantial amount of depleted batteries due to replacements. Users have a major impact on battery health degradation through their actions while handling electric vehicles, such as the use of fast charging. To mitigate this problem, this article presents a methodology to generate user guidance for batteryfriendly actions in the upcoming use. Therefore, we first identify general recommendations from related work and combine them with the vehicle context in order to define context-aware recommendations in the form of if–then rules. These context-aware recommendations are then used to generate user advice. Second, the article covers how to predict the vehicle context in order to determine necessary recommendations. Third, a prescriptive recommendation system architecture is proposed, which takes vehicle context information, and produces user guidance. Finally, we test the proposed architecture using fuzzy logic as decision system. Overall, the architecture provides satisfactory user advice

  @article{eider2023context, abstract = {Electric vehicles are a means of reducing CO2 emissions in transportation. However, the sustainability of electric vehicle batteries is affected by battery health degradation, which decreases their overall lifetime. This results in a substantial amount of depleted batteries due to replacements. Users have a major impact on battery health degradation through their actions while handling electric vehicles, such as the use of fast charging. To mitigate this problem, this article presents a methodology to generate user guidance for batteryfriendly actions in the upcoming use. Therefore, we first identify general recommendations from related work and combine them with the vehicle context in order to define context-aware recommendations in the form of if–then rules. These context-aware recommendations are then used to generate user advice. Second, the article covers how to predict the vehicle context in order to determine necessary recommendations. Third, a prescriptive recommendation system architecture is proposed, which takes vehicle context information, and produces user guidance. Finally, we test the proposed architecture using fuzzy logic as decision system. Overall, the architecture provides satisfactory user advice}, author = {Eider, Markus and Sick, Bernhard and Berl, Andreas}, journal = {Sustainable Computing: Informatics and Systems (SUSCOM)}, keywords = {isac-www}, pages = 100845, publisher = {Elsevier}, title = {Context-aware recommendations for extended electric vehicle battery lifetime}, volume = 37, year = 2023 }