Real-Time Multilingual Verbal Command Recognition for Safety Protocols in Power Transmission and Distribution Systems

https://doi-002.org/1018/17606854644308


Yongjiao YANG 2、Qiang QIN * 3、Jiaxin LIN 4、hanyeHUANG 5、ruiqiLI

Affiliation: Information Center, Guangdong Power Grid Co., Ltd., Guangzhou, Guangdong 510000,China
1、Email:meigui10203040@163.com
2、Email:qin201709092025@163.com
3、Email:yu20041128fjzdf@163.com
4.qun19860116@163.com
5、 li9830507@163.com
Corresponding Author:Qiang QIN
ABSTRACT
Electricity transmission networks are critical infrastructures where timely fault detection is essential to ensure stability, reliability, and safety. Adaptive human-machine interaction offers a new paradigm to enhance situational awareness and decision-making in real-time fault monitoring. Traditional methods often rely on static rule-based protection schemes or centralized algorithms that face challenges in handling streaming data, dynamic grid conditions, and operator cognitive overload. These limitations lead to delayed responses, false alarms, and reduced trust in automation. To address these issues, this paper proposes an Adaptive Human-in-the-Loop Fault Detection (A-HIL-FD) framework, which integrates real-time data analytics, physics-aware anomaly detection, reinforcement learning, and adaptive operator interfaces. The framework dynamically adjusts automation levels, provides explainable fault localization outputs, and incorporates operator feedback to continuously refine decision-making. The proposed method enables operators to receive context-aware fault alerts, confidence scores, and actionable recommendations via adaptive visualization, thus improving fault detection accuracy and reducing response latency. The proposed A-HIL-FD framework demonstrates superior performance across eight key parameters: Fault Detection Accuracy (97), False Alarm Rate (4), Mean Time to Detect (55 ms), Operator Cognitive Load (50), Decision Confidence (95), Adaptive Automation Level (70), Root Cause Localization Accuracy (88), and System Resilience Score (95).
Keywords: Adaptive Interaction, Fault Detection, Human-In-The-Loop, Reinforcement Learning, Electricity Transmission, Real-Time Monitoring.
INTRODUCTION
Today’s power systems’ electrical transmission infrastructure ensures that electricity from power plants reaches distribution networks and consumers securely [1]. It is critically important that these networks continue to function well and without issues because flaws or interruptions may lead to enormous economic losses, widespread power outages, and equipment damage [2]. The integration of renewable energy sources, distributed generation, and advanced grid technology has increased the dynamics and complexity of transmission networks [3]. Because of this, it has become much harder to find faults and keep an eye on systems [4]. Most traditional ways of finding and fixing problems in transmission networks use centralized monitoring algorithms, relays that work based on thresholds, or rules that have already been set [5]. Even though these methods work sometimes, they have built-in problems when it comes to handling adaptive system behaviors, real-time streaming data, and unexpected fault scenarios [6]. They tend to create response delays or alarms falsely, which may be more than human operators can process, and cause them to think harder [7]. This may even reduce people’s trust in automated security systems [8]. To handle the increased sophistication of today’s grids, the system requires new methods to integrate human expertise with automatic fault detection in a manner that can self-correct [9]. Recent breakthroughs in AI, data analytics, and human-machine interaction offer promising answers to these constraints [10]. Adaptive human-in-the-loop systems take advantage of the fact that humans and computers collaborate effectively [11]. These systems allow the process to create sensible alerts, process data in real time, and apply operator interfaces that are easy to comprehend to continue refining the intelligence of the system [12]. To dynamically alter the automation levels in real time. To strengthen the system’s ability to learn from operating experience and enhance fault detection accuracy, optimization of decision-making policies over time can be achieved, especially by means of reinforcement learning. In the following sections, the paper discusses in detail the design of the proposed framework, the learning and problem-finding algorithms utilized, and an exhaustive analysis of how well the framework performs under realistic transmission network conditions. These results indicate that adaptive human-machine interaction can assist with issues of existing grid monitoring and operational decision-making.
Table 1: Overview of Problem, Limitations, and Proposed Solution
Aspect Description
Domain Electricity transmission networks are critical infrastructure requiring real-time fault detection.
Problem Statement Faults in transmission systems threaten stability, reliability, and safety; timely detection is essential.
Limitations of Existing Methods Static, rule-based protection schemes – Centralized algorithms struggling with streaming and dynamic grid data – Cognitive overload for human operators – High false alarm rates and delayed responses
Research Gap Lack of adaptive, explainable, and human-centered fault detection frameworks that combine automation with operator feedback
Proposed Framework Adaptive Human-in-the-Loop Fault Detection (A-HIL-FD) integrating real-time data analytics, physics-aware anomaly detection, reinforcement learning, and adaptive operator interfaces
Expected Contributions – Context-aware alerts with confidence scores

  • Adaptive visualization for situational awareness
  • Reduced false positives and response latency
  • Increased operator trust in automation
  • Enhanced resilience and efficiency of electricity transmission networks PRELIMINARIES
    Recent technologies in adaptive fault detection, human-machine interface, and Industry 5.0 have contributed significantly to resilient power systems. Prior research has explored adaptive interfaces, multi-agent optimization, machine learning-based fault detection, and huge language model-based frameworks. These works demonstrate that integrating automation with human-in-the-loop intelligence is the most effective means of stabilizing the grid, establishing trust with operators, and accelerating decision-making.
    Table 2: Related works summary
    Ref Focus Area Methods/Techniques Key Contributions Relevance to Proposed System
    [13] Sahu et al. (2023) Adaptive fault diagnosis in active distribution networks Machine learning–based adaptive fault diagnosis considering hosting capacity (ML-AFD-HC) Improved fault detection under varying hosting capacity conditions in distribution systems Shows adaptive ML approaches for fault diagnosis in smart grids
    [14] Mazhar et al. (2025) Smart grid + Industry 5.0 integration Conceptual framework and survey of Industry 5.0 applications Identifies challenges and solutions for smart grid integration with Industry 5.0 technologies Provides contextual background on human-centric smart grids
    [15] Jain et al. (2024) Adaptive human–machine interaction in ubiquitous networks Adaptive interaction design and ubiquitous network-based HMI Proposed adaptive HMI for ubiquitous computing environments Links adaptive HMI concepts to distributed power networks
    [16] Jin et al. (2024) Power grid decision agents + human–machine collaboration Multi-agent optimization and dynamic collaboration framework (PGDA-HMC) Framework for decision support in grid optimization using human–machine collaboration Aligns with grid decision support & HIL optimization
    [17] Xu et al. (2024) HMI devices (sensors) Flexible pressure sensors for advanced HMI applications Advances in sensor design for more responsive human–machine interfaces Hardware perspective: enables more accurate operator interaction in fault detection
    [18] Hamdani & Chihi (2025) Adaptive HCI in fault detection and diagnosis Human-centric adaptive interaction design for Industry 5.0 Integration of adaptive HCI to improve fault detection in Industry 5.0 systems Directly supports adaptive HMI for fault diagnosis frameworks
    [19] Tong (2025) Human–robot interaction (manufacturing) User experience design and cognitive evaluation techniques Enhances operator experience in manufacturing environments Secondary relevance: contributes to user trust and workload balance in HMI
    [20] Feng et al. (2025) Fault management in subway power supply systems Multi-agent systems + AI for fault diagnosis, isolation, and recovery (MAS-AIFDR) Self-healing strategies for power supply systems Relevant for automated fault handling in large-scale networks
    [21] Zhang et al. (2025) Industrial human-in-the-loop fault diagnosis Large Language Model–based Time Series Fault Diagnosis (LLM-TSFD) Introduces LLM-assisted explainable HIL fault detection Strongly relevant: demonstrates human-in-the-loop + AI in fault detection
    [22] Nderitu (2023) Long-term adaptive interfaces Mental state modeling for adaptive interfaces Framework for adapting to user mental states in prolonged HMI sessions Adds a dimension of operator cognitive adaptation to fault detection systems

There are numerous means of adaptive monitoring and interaction within power systems, as indicated in the literature in the above Table 2. Some of these include cognitive and mental-state adaptive interfaces, multi-agent coordination, and ML-enabled defect detection. Finally, the results of this study demonstrate that hybrid human-machine systems can significantly improve the speed, accuracy, and insightfulness of defect detection. The identified A-HIL-FD architecture can proceed with optimism now that this solid foundation has been established.
NOTATIONS AND MATHEMATICAL BACKGROUND
The mathematical notations, symbols, and operations are utilized throughout the remainder of the text. The ensuing equations and algorithm descriptions will be more understandable since they present a uniform foundation for representing the Adaptive Human-in-the-Loop Fault Detection system. They consist of representations as vectors and matrices, sensor readings, anomaly scores, operator input, and reinforcement learning parameters. Equation 1 involves the fusion of numerous sensor measurements into a single state estimate for reliable and robust fault detection in real time.
X_t= ∑_(i=1)^N▒〖w_i* S_t[i] 〗 (N)+ ε_t (1)
X_t is the combined system state vector at time t, N refers to the sum of the total number of sensors or data sources, w_i is the reliability weight on behalf of the ith sensor, S_t[i] is the sensor measurement vector from the sensor at time t, and ε_t is the random noise or uncertainty in the measurements. The model parameters are updated based on operator feedback to enhance action-adaptation capabilities using equation 2.
Θ_t1= Θ_t+ α * F_t⊙ (X_t- Θ_t ) (2)
Θ is the styled feedback vector at time t, α is the set learning rate determining the influence of the operator feedback, F_t is the operator feedback vector 1 for a correct action, and 0 otherwise, ⊙ indicates element-wise multiplications, X_t is the fused system state vector [22]. Equation 3 is used to determine the probability of selecting the action associated with optimal RL-based fault mitigation.
P(at │Xt)=exp⁡(Q(X_t,a_t ))/(∑_a▒exp⁡(Q(X_t,a)) ) (3)
P(at │Xt) is the probability of taking action at a given state X_t, Q(X_t,a_t ) is the expected reward of taking action at a given state X_t, a represents all possible actions.

Figure 1: Process of Sensor Measurements Standardization
Equation 4 standardizes sensor measurements to reliably detect outlying values between heterogeneous data sources.
Z_t[i] =((S_t[i] – μi ))/σ_i (4) Z_t[i] is the standardized measurement vector of instrument i at time t, S_t[i] is the instrument’s raw measurement. μ_i and σ_i are the mean and standard deviation of the measurement instrument’s historic measurements are given in Figure 1. Equation 5 is used as a measure to identify deviations from physics-informed behavior and identify anomalies consistent with the physics of the grid [32]. R_t= Y_t- f_phys (X_t,u_t ) (5) R_tis the residual at time t, where (X_t,u_t ) from physics-informed predictions, Y_t is the control input of the system, and f_phys (X_t,u_t ) is the given state X_t and control input u_t. Equation 6 produces a unified anomaly score that incorporates both statistical and physics-based indicators. A_t= ∑(i=1)^N▒βi (Z_t[i] )+γ * ‖R_t ‖^2 (6) A_t is the total anomaly score at time t, N is the number of sensor channels, β_i is the weight for the normalized sensor deviation (Z_t[i] ), γ is the weight for the physics-based residual R_t,〖 ‖.‖〗^2 is the Euclidean norm [23]. Equation 7 provides the context in which the RL agent recognizes the state to make a decision. s_t= X_t,A_t,O_t s_t represents the state vector of the RL agent at time t, X_t denotes the fused state of the system, A_t is the anomaly score, and O_t is a representation of operator feedback or observations. Equation 8 guides the RL agent to maximize correct actions while minimizing false alarms and delays. r_t= λ1 * 1_correct- λ2 * 1(false_alarm )- λ3 * Δt (8)
r_t is the reward at time t correct equals 1 if the RL action matches correct fault mitigation, 1 false_alarm equals 1 for incorrect alarms, Δt is the response delay, λ1,λ2,λ3 are weighting coefficients [33].

Figure 2: Process of RL Agent’s Policy
Equation 9 updates the RL agent’s policy to improve decision-making in the long term through reward.
Q(s_t,a_t )= η * (r_t+ γ *max⁡(Q(s_(t+1),a))- Q(s_t,a_t )) (9)
Q(s_t,a_t ) represents the action-value function, η is the learning rate, γ is the future rewards discount factor, r_t is the reward, s_(t+1)is the next state, and max max⁡(Q(s_(t+1),a)) is the expected future value as shown in Figure 2. Equation 10 estimates alert confidence levels based on system detection and operator feedback [24].
C_t=1/A_t (exp⁡(-κ * F_t ) ) (10)
C_t refers to the alert confidence level at time t, A_t represents the anomaly score, F_t is operator feedback (1 for confirmed, 0 otherwise), κ is a scaling factor that controls the influence of feedback on alert confidence. Equation 11 enables operator workload to be indexed to adaptively adjust interface complexity.
W_t= φ1 * N_alerts+ φ2 *∑_(i=1)^(N_actions)▒(a_i ) (11)
W_t is the estimated cognitive workload, N_alerts is the number of alerts that are active at the same time, N_actions is the number of actions an operator takes, a_i is the effort taken per action, φ1 and φ2 are the weighting constants. The current configuration of the interface is denoted by I_t1, the interface elements will be adjusted to improve the usability of the interface in equation 12 and situational awareness based on operator feedback.
I_t1= I_t+ δ * (F_t- I_t ) (12)
where F_t is the operator feedback vector, δ is the adaptation rate, denoted by δ, I_t controls the speed of interface adaptation. By introducing distinct mathematical rules and notations, the present section provides the foundation for accurate calculations of sensor fusion, anomaly detection, reinforcement learning updates, and interface adaptation. It assists in thoroughly testing the performance of the proposed A-HIL-FD system in identifying faults in electrical transmission networks and ensures that every equation and algorithm is identical, thus simplifying the use and understanding of the system [34].
PROPOSED PROTOCOL
Figure 3 depicts an architecture that integrates various data sources, analytics engines, reinforcement learning, and operator interfaces to enable real-time management of faults in electrical transmission networks. Sensor data streams, grid historical analysis datasets, and simulation models are the most critical components of system monitoring.

Figure 3: The A-HIL-FD Architecture
The anomaly detection and data analytics module utilizes such inputs and applies statistical and physics-based techniques to identify defects. The Adaptive Decision Engine, which is Reinforcement Learning-based, receives the outputs and employs them to make runtime adjustments to its automation and decision-making levels according to what the operator dictates and how things shift. Continuous integration of feedback refers to the fact that the operator’s decisions immediately impact the system’s learning. The final component, the adaptive human-machine interface, informs operators more about what is happening and accelerates their response by providing them with context-aware alerts, transparent graphics, and actionable recommendations [25]. The corresponding design safeguards valuable power transmission assets and ensures that they are efficient, reliable, and robust.
Algorithm 1: The A-HIL-FD for Real-Time Grid Monitoring
Input:
St=[〖St〗1,〖St〗_2,…,〖St〗_n ] — Sensor measurements, Ft — Operator feedback vector, Θt — Current model parameters, H — Number of attention heads Output: P(a_t∣X_t) — Action probabilities, Θ(t+1) — Updated model parameters

1. Sensor Fusion

X_t= sum(w_i* S_t[i] for i in range(N))+ ε_t # ε_t is measurement noise

2. Multi-Head Attention

A_heads= [N]
for h in range(H)
Q_h= X_t@ W_Q[h] # Query matrix
K_h= X_t@ W_K[h] # Key matrix
〖 V〗_h= X_t@ W_V[h] # Value matrix
A_h= softmax(Q_h@ K_h.T/√((d_k ) ))@ V_h
A_heads.append(A_h )
A_t= concat(A_heads )@ W_O # Concatenate multi-head outputs

3. Operator Feedback Integration

Θ_t1= Θ_t+ α * F_t* (X_t- Θ_t )

4. Compute Action Probabilities

P_a= softmax(Q_func(X_t ) )

5. Decision Selection (optional)

if max⁡(P_a )> τ
a_t= argmax(P_a )
else
request_(operator_((confirmation) ) )

The proposed A-HIL-FD process utilizes architecture to assist power grids in detecting faults that evolve over time. It is a hybrid combination of real-time sensor indication and operator input. The first step in Algorithm 1 integrates sensor readings from multiple sensor types S_t[i] and merges them to develop a single system state vector X_t utilize a weighted summation with a noise factor to account for uncertainty. Next, there is a multi-head attention process, whereby each head produces a query Q_h, key K_h, and value K_h matrix, which generates an attention score through a scaled dot product, and which creates feature embeddings. These are then merged, providing an enriched, complete state picture through attention a_t. Feedback from operators (Ft) provides the capability to update model parameters: Θ,b,and t. This creates capabilities for customizable education. Finally, a Q-function based on reinforcement learning yields action policy probabilities (Pa) to mitigate the possibility of mistakes. The optional choice selection is based on a threshold (τ), as to either begin automatic action, or to ask the operator to consent to action. In turn, this methodology, which balances automation and human control, leads to increased real-time situational awareness, decreased false positives, and improved decision-making capabilities in the dynamic energy transmission network.

Figure 4: Illustration of applying physics to examine data in real time
The examination and locating unusual things in Figure 4 ensure that anomalies are detected with the help of physics models and real-time data analysis combined. This is crucial for detecting faults [26]. Data streaming from SCADA, IoT sensors, and Phasor Measurement Units (PMUs) needs to be pre-processed to eliminate noise and normalize signals. Next, the data is then processed through two additional techniques: statistical models and physics-based rules. Statistical models are able to monitor trends over time, detect abrupt change, and detect anomalous behavior in power systems. Physics-based rules derived from grid equations and dynamic models verify fundamental electrical laws, such as load flow and Kirchhoff’s laws, simultaneously. The two outputs are combined in an anomaly fusion and scoring engine, where they compare signals from multiple sources, make an educated guess about how severe an event is, and assign each event a confidence level. With the addition of dependability scores to flagged events, this process allows operators and decision-making systems to place top-priority high-risk anomalies on their lists. The corresponding fusion ensures that anomalies are data-driven and conform to physics and assists in reducing false alarms.
Table 3: Real-time Fault Detection Performance
Metric Traditional Method A-HIL-FD Improvement (+/-)
Fault Detection Accuracy (FDA) 82 97 +15
False Alarm Rate (FAR) 14 4 -10
Mean Time to Detect (MTTD, ms) 160 55 -105
Decision Confidence (DC) 75 95 +20
Operator Cognitive Load (OCL) 80 50 -30

Table 3 indicates that the A-HIL-FD framework performs better than the conventional methods of fault detection in real time. An improvement of fifteen points in the accuracy of fault detection indicates that applying physics to detect anomalies is an effective method for detecting real issues. A reduction by a factor of ten in the number of false alarms implies that there are significantly fewer false positives. A reduction in the Mean Time to Detect by 105 ms implies that real-time analytics of data makes humans respond quicker. Improved alert prioritization implies that operators do not need to think as much and are more confident in their actions, both of which indicate that the predictions of the system are more accurate [27]. Overall, these enhancements demonstrate that applying physics-based validation to streaming analytics simplifies and makes it more accurate to identify issues.
The reinforcement learning-based adaptive decision module is the intelligent control layer of the A-HIL-FD system, as shown in Figure 5. It allows fault detection and response techniques to continually improve. The state representation illustrates the manner in which the system is presently operating. It consists of input from anomaly scores, grid conditions, and operator feedback. The actions of the RL agent are determined by this state, such as altering levels of automation, recommending how to isolate things, or altering the level of complexity of the visualization [28].

Figure 5: Reinforcement Learning-Based Adaptive Decision Module
The outcome is judged using a reward feedback engine that considers both the operators’ subjective evaluation and objective system performance metrics. The model points toward time-optimal strategies by punishing false positives or wasteful actions, and those that are correct. This feedback loop enables the decision policies to adapt according to shifting grid situations and operator requirements. This module makes it more responsive, reduces some of the burden on operators, and ensures that decisions are clear, fast, and consistent with how the transmission grid actually operates by bridging automated processes with human monitoring.
Algorithm 2: Real-Time Data Analytics and Physics-Aware Anomaly Detection
Input:
〖St〗_i — Sensor measurements for i=1,…,N , Xt — Fused system state, Physics model f_phys (⋅)
Output:
At — Anomaly score
Rt — Physics-based residuals

1. Data Normalization

Z_t= [(S_t[i] – μ_i)/(σ_i for) i in range(N)] # Normalize each sensor

2. Compute Physics-Based Residuals

R_t= Y_t- f_phys(X_t,u_t ) # Deviation from expected physics

3. Anomaly Score Calculation

A_t= sum(β_i* abs(Z_t[i] )for i in range(N))+ γ * norm(R_t,2)

4. Thresholding

if A_t> threshold
flag_((anomaly) )
else
continue_monitoring()

The above algorithm 2 refers to the real-time data analytics and physics-aware anomaly detection plugin for the A-HIL-FD framework. The first thing accomplished is to make all sensor readings consistent 〖St〗_i by averaging over time μ_i, and the standard deviation σ_i can bring them to an agreement with sensor readings from other sources. After this step, a model based on physics f_phys(X_t,u_t ) was created to compute R_t comes up with the expected output of the system. Meanwhile, the residual Z_t[i] computes a number representing how different the sensor data is from the expected output [29]. Collectively, these two variables, called statistical deviations Z_t[i] will be referenced later, are expected to work together. The leftovers are all combined into a single strange score A by plugging in a weight system β_i. The value of A can flag an anomaly at an abnormal level A_t suppose it is greater than a threshold limit. The two-pronged approach provides reduced false alarms while still maintaining that an abnormality represents a real meaning. This lays the groundwork leading to adaptive decision making and operator assistance later on.

Figure 6: An Adaptive Operator Interface and a Feedback Loop
The adaptive human-machine interface, linking the operator with the automatic system, ensures that fault detection is clear and reliable. The interface consists of two principal components, given in Figure 6: the operator interface console and the panels displaying faults. A few of the visualization tools that assist operators in comprehending the system status include topology diagrams, stress zone heatmaps in real time, and event timelines. The operator is able to respond to alarms, alter the suggestions of the system, and provide feedback via the interface. The explainable AI output layer is the most critical portion of the interface. It provides unambiguous root cause identification, confidence levels, and actionable suggestions. By continuously feeding this data back into the reinforcement learning engine, decision-making will progressively improve over time [30]. The balance of automation and operator interaction in the interface contributes to decreased cognitive load, accelerated reactions, and operator trust. It finally offers good fault management by integrating real-time situational awareness and adaptive decision support.
Table 4: Adaptive Human-in-the-Loop Interaction
Metric MAS-AIFDR A-HIL-FD Improvement (+/-)
Adaptive Automation Level (AAL) 55 70 +15
Root Cause Localization Accuracy (RCLA) 83 88 +5
System Resilience Score (SRS) 86 95 +9
Operator Cognitive Load (OCL) 55 40 -15
Decision Confidence (DC) 88 96 +8

Here is how the adaptive operator interface and feedback loop function, as presented in Table 5. The 15-point rise in Adaptive Automation Level illustrates the impact of human input on how the system operates. Locating the Sources The physics-informed models’ capability to identify faults immediately results in a fivefold improvement in accuracy. The increase in System Resilience Score by 9 points indicates that the system is more reliable to operate. The interface is successful in distributing attention and reducing stress, as seen by a decrease in operator cognitive load by 15 points [31]. Placing greater importance on trustworthy notifications and suggestions can boost decision confidence by 8 points. The findings indicate that integrating adaptive interfaces and operator feedback within real-time fault management enhances situational awareness, operational effectiveness, and human-system interaction overall.
EXPERIMENTAL ANALYSIS
The outcome analysis compares the proposed A-HIL-FD framework with simple and complex approaches, i.e., PGDA-HMC, ML-AFD-HC, and MAS-AIFDR. The objective of this systematic review is to demonstrate that real-time fault detection and human-in-the-loop integration are superior by examining critical metrics such as mean time to detect, decision confidence, root cause localization, adaptive automation, false alarm rate, accuracy, and system resilience.
DATASET DESCRIPTION
The data set encompasses normal conditions and fault cases for 110 kilovolt transmission lines. It contains 24 features, with one being the capability to display the current and voltage levels when there is a short circuit across various busbars per phase. The fault can occur anywhere between 0% to 100% of the line’s length. The three types of faults are single-phase-to-ground, two-phase, and three-phase short circuits. Tags indicate the type of fault, faulty line, and location where the fault occurs to assist in locating issues and making choices in real-time [35].
Table 5: Dataset Description
Attribute Description
Dataset Type Simulated 110 kV power transmission line data
Features 24 variables, including voltage magnitudes and short-circuit currents for each phase at busbars
Fault Types Single-phase-to-ground, two-phase short circuit, three-phase short circuit
Fault Locations 0% to 100% along the transmission line
Normal Operation Baseline data without faults
Labels / Target Variables Fault type, faulted line, and fault position
Purpose Supports real-time anomaly detection, physics-aware modeling, and adaptive decision-making

Table 6: Simulation Environment
Parameter Description
Simulation Platform MATLAB/Simulink or Python-based power system simulation environment
Network Type 110 kV power transmission line network
Fault Types Simulated Single-phase-to-ground, two-phase, three-phase short circuits
Fault Locations 0% to 100% along the transmission line
Time Step / Sampling Rate 1 ms – 10 ms (configurable) for real-time analysis
Number of Buses / Nodes Multiple buses with voltage and current monitoring per phase
Sensor Data Voltage magnitude, current magnitude, phase angles
Operator Interaction Human-in-the-loop feedback simulated via adjustable input signals
Reinforcement Learning Q-learning or Deep Q-Network for adaptive decision-making
Data Storage / Logging CSV/Mat files for each simulation run, including both normal and fault conditions
Purpose To test real-time fault detection, anomaly scoring, and adaptive interface response

FAULT DETECTION ACCURACY (FDA)

Figure 7: Fault Detection Accuracy Analysis
Figure 7 indicates that the A-HIL-FD framework consistently achieves the optimal detection accuracy across all problem types. Detection is as high as 94–98% with A-HIL-FD. Other sophisticated techniques (PGDA-HMC, ML-AFD-HC, MAS-AIFDR) take it to 90–92, and traditional techniques take it to 70–85. The findings indicate that the integration of real-time analytics with physics-based anomaly detection and human-in-the-loop feedback ensures it becomes much simpler to identify faults, detect missed detections, and maintain consistent performance in simplistic and complicated fault scenarios using Equation 13.
Equation 13 quantifies weighted fault detection accuracy by conditioning on prediction, feedback, and physical residuals.
FDA_t= (1/N_f ) ∑_(i=1)^(N_f)▒1 ([(argmax(Pa|X_t,Θ_t,F_t )== a_true[i] ) .exp⁡(-λ ‖R_t ‖) ]) (13)
Fault Detection Accuracy FDA_t evaluates the percentage of faults correctly detected at time t. Here, N_f is the overall count of faults that occurred, X_t is the sensor observation vector, Θ_t is the operator-set threshold of decision, and F_t is the feedback vector by the human monitor. The a_true[i] denotes the actual type of fault, while the (Pa|X_t,Θ_t,F_t ) is the predicted probability of each fault type. ‖R_t ‖ is the physics-based residual, and λ scales its effect to take into account model prediction, human feedback, and physics validation.
FALSE ALARM RATE (FAR)
Table 7: False Alarm Rate Comparison
Fault Type PGDA-HMC ML-AFD-HC MAS-AIFDR A-HIL-FD
Single-phase-to-ground 10 8 6 3
Two-phase short circuit 11 9 7 4
Three-phase short circuit 12 10 8 5
Phase-to-phase-to-ground 11 9 7 4
No-fault / Normal 4 3 2 1

The FAR table 7 indicates that A-HIL-FD reduces false alarms significantly compared to other approaches. MAS-AIFDR reports fewer false alarms for each scenario 6 to 8. PGDA-HMC and ML-AFD-HC reduce it to 8 to 12, and conventional techniques report 12 to 15. A-HIL-FD indicates that it is more trustworthy by reporting only 1–5. The system integrates operator input with adaptive anomaly scoring to minimize the workload of the operator by equation 14, enhance the reliance on automated problem detection, and prevent false alarms. It is more effective than ML-only and static approaches. Equation 14 is to measure false positives weighted by confidence scores and improve reliability.
FAR_t= (1/N_s )* ∑_(i=1)^(N_s)▒〖(1-1 (〖fault〗_i∧ (fault) ̂_i)) 〗.( σ(C_t- τ)) (14)
The FAR_t measures how often fault alerts are incorrect N_s is the number of trials, 〖fault〗_i and (fault) ̂_i are the observed and predicted faults, C_t is the confidence score of the alert, τ is the confidence threshold σ is a sigmoid function, which penalizes false positives based on the confidence of the alert, and reflects the probability that the alert is false.
MEAN TIME TO DETECT (MTTD)

Figure 8: Mean Time to Detect Analysis
A-HIL-FD is the fastest method to detect various types of faults shown in Figure 8. MAS-AIFDR achieves it in 85–100 ms, PGDA-HMC and ML-AFD-HC reduce it to 95–135 ms, while conventional methods take 140–170 ms. A-HIL-FD is more real-time responsive since it can identify things in 45 to 60 ms. This improvement is extremely crucial since early fault detection prevents failure spreading in power transmission by Equation 15. A-HIL-FD is able to rapidly detect issues even in complex cases with various stages or extended distances since it integrates real-time sensor analytics with adaptive decision-making. Equation 15 computes fault detection latency, holding human-in-the-loop contributions constant.
MTTD = (1/N_f ) ∑_(i=1)^(N_f)▒〖(t_detect[i] – t_fault[i] ).〗 (1 + α * ‖N_f ‖) (15)
MTTD computes the average time to detect faults, t_fault[i] is when the fault actually occurred, t_detect[i] detect is when the fault was detected, α is a scaling factor for operator influence, and ‖N_f ‖ is the magnitude of human feedback, which captures the influence of operator intervention and the significance of detection response effort in real-time.
OPERATOR COGNITIVE LOAD (OCL)
Table 8: Operator Cognitive Load Comparison
Automation Level PGDA-HMC ML-AFD-HC MAS-AIFDR A-HIL-FD
0 80 75 70 55
25 75 70 65 50
50 70 65 60 45
75 65 60 55 40
100 60 55 50 35

The OCL table 8 illustrates that A-HIL-FD simplifies operators’ work at all levels of automation. The conventional approach loads people the most (65–85), whereas MAS-AIFDR reduces it to 50–70. A-HIL-FD always reduces the load to 35–55 using operator feedback, selective alarms, and adaptive visualization. When the contrast human-in-the-loop interaction with fully automated or ML-only configurations and using equation 16, it can be observed that the former involves a lower cognitive load. This accelerates decision-making, reduces fatigue, and allows operators to concentrate on significant events.
Equation 16 quantifies operator mental workload based on alerts and automation interaction.
OCL_t= ∑_(i=1)^(N_a)▒〖w[i].log⁡(1+〗 n_i (t))+ β ‖A_t- I_t ‖ (16)
OCL_t evaluates mental workload on operators, N_a is the simultaneous number of alerts, w[i] is the type weight of alert, n_i (t) is type frequency, A_t is an automation output vector, I_t is operator input, β scales the mismatch effect, the equation serves as a description/measure of alert complexity, and HIL interaction.
DECISION CONFIDENCE (DC)

Figure 9: Decision Confidence Analysis
A-HIL-FD boasts the highest degree of decision confidence across all automation levels (92–97) compared to traditional methods (70–78) and MAS–AIFDR (82–90) in Figure 9. The integration of confidence scoring, operator feedback, and explainable defect outputs ensures decisions are robust and understandable through equation 17. By increasing confidence, the method was able to outperform older approaches that provided constrained or ambiguous decision guidance. This made operators rely on automated suggestions, accelerated response, and made the system safer in general, where DC assesses the degree of confidence in predicted faults. Equation 17 produces confidence scores by combining model probability DC_t with the severity of the anomaly.
DC_t=(∑(i=1)^(N_f)▒[P[a_i ][X_t ].exp⁡(-γ *‖R_t ‖) ] )/(∑(i=1)^(N_f)▒[P[a_i ][X_t ]] ) (17)
P[a_i ][X_t ] is the model probability of predicting fault i, ‖R_t ‖ is the residual from physics-aware anomaly detection, γ represents the weight on the severity of anomalous detection, and N_f is the number of fault types. The numerator observes penalizing uncertain or physics mixed predictions, and the denominator is simply a normalization for the score.
ADAPTIVE AUTOMATION LEVEL (AAL)
Table 9: Adaptive Automation Level Comparison
Time Step PGDA-HMC ML-AFD-HC MAS-AIFDR A-HIL-FD
1 25 30 35 50
2 30 35 40 55
3 35 40 45 60
4 40 45 50 65
5 45 50 55 70

The AAL table 9 indicates A-HIL-FD enhances automation throughout the simulation steps, peaking at Step 5. This compares to MAS-AIFDR (55), ML-AFD-HC (50), and A-HIL-FD (70). There have not been a lot of changes to the traditional ways of doing things (20–40). The system can dynamically alter to balance autonomous operation and human intervention to optimize efficiency and ensure operators are engaged in critical scenarios by equation 18. It demonstrates that A-HIL-FD is able to alter its automation in real time in accordance with system changes and operator input. Equation 18 evaluates real-time adaptive automation balancing human and system input.
AAL_t= (1/N_s )*∑_(i=1)^(N_s)▒〖(Θ[i].ηF[i]).exp⁡〖(-δ .‖R[i]‖) (18)〗 〗
AAL_t indicates the percentage of system operations performed automatically, Θ[i] is the base automation threshold, F[i] is operator feedback, η is a scaling of feedback, ‖R[i]‖ is the residual error, δ is a weighting of residual impact, and N_s is the total state of the system, and the process modifies automation online using feedback and system trust.
ROOT CAUSE LOCALIZATION ACCURACY (RCLA)

Figure 10: Root Cause Localization Accuracy
Figure 10 for RCLA illustrates that A-HIL-FD is far superior in determining the precise location of faults. MAS-AIFDR and ML-AFD-HC both process 81–87 and 78–85 for 0–100% line positions, while A-HIL-FD processes 85–94. Proper localization is required for timely maintenance and fault-solving. A-HIL-FD is superior to conventional and ML-only approaches since it can detect anomalies through physics and employ reinforcement learning to minimize diagnostic mistakes and allow individuals to make optimal operational choices by equation 19.
Equation 19 evaluates the precision of combined fault type and location identification.
RCLA = (1/N_f ) ∑(i=1)^(N_f)▒〖exp⁡(-‖L_pred[i] – L_true[i] ‖^2.1(T_pred[i] == T_true[i] ) 〗 (19) RCLA assesses the accuracy of fault localization, the terms L_pred[i] and L_true[i] represent the predicted and true locations of the fault, while T_pred[i] and T_true[i] represent the predicted and true fault types, N_f is the number of faults, and 〖∥⋅∥〗^2 is the Euclidean distance. The exponential function penalizes errors in localization, while the indicator assures that the fault type is categorized correctly. SYSTEM RESILIENCE SCORE (SRS) Table 11: System Resilience Score Comparison Time Step PGDA-HMC ML-AFD-HC MAS-AIFDR A-HIL-FD 1 70 75 78 90 2 72 76 80 91 3 74 78 82 92 4 76 80 84 94 5 78 82 86 95 In resilience in unpredictable situations, A-HIL-FD performs better compared to other approaches, which rate 65–86 in Table 11. Increasing SRS values indicate that fault detection, operator feedback, and adaptive automation have been more integrated. The design is more resilient, scalable, and superior to ML-based and MAS-based approaches because it repeatedly minimizes false alarms and responds swiftly to defects by equation 20. Equation 20 is the composite metric analysis that captures the overall reliability and responsiveness of the system. SRS_t= ∑(i=1)^(N_f)▒〖w_1〖FDA〗_i- w_2〖FAR〗_i- w_3〖MTTD〗_i+ w_4〖DC〗_i 〗 (20)
〖FDA〗_i, 〖FAR〗_i, 〖MTTD〗_i, and 〖DC〗_i are the fault detection accuracy, false alarms, detection time, and decision confidence for the fault i w_1, w_2, w_3, w_4 are weighting factors and N_f is the total fault occurrences. SRS combines several performance metrics into one metric that captures the resilience of the system. All the performance measures state that A-HIL-FD performs superior to the present methodologies. The work of the framework has resulted in improved system robustness, improved fault detection accuracy, reduced false alarms, improved detection speed, reduced operator cognitive workload, increased decision confidence, more flexible automation, and improved root cause localization. This indicates that mixing human input with analytics that account for physics as well as reinforcement learning is effective for improved power transmission fault management.
DISCUSSION OF LIMITATIONS, CONCLUSION, AND FUTURE WORKS
5.1 LIMITATIONS
The A-HIL-FD framework, as proposed, greatly enhances the accuracy of fault detection, response time, and operator decision-making; however, there are some limitations. First, the present arrangement is based on high-quality streaming sensor readings, which might fail if there are missing measurements or the measurements contain noise. Second, it might be difficult to maintain the operator engaged for extended durations or under high-stress conditions due to the human-in-the-loop aspect. Third, the reinforcement learning component must have historical data to function. If novel kinds of faults suddenly appear, performance will decline for a short period until sufficient feedback has accumulated. Finally, extremely large-scale transmission networks could require optimal parallel processing or custom hardware to deploy in real-time since the scheme is so computationally expensive.
5.2 CONCLUSION
The proposed method demonstrates adaptive operator interfaces for power transmission grids, real-time data analytics, anomaly detection with a physics-aware module, reinforcement learning, and an A-HIL-FD framework. A-HIL-FD outperforms conventional, PGDA-HMC, ML-AFD-HC, and MAS-AIFDR approaches in several key performance metrics, including adaptive automation, accuracy in fault detection, false alarm rate, mean time to detect, cognitive load on the operator, confidence of decision-making, and general system resilience. The findings indicate that the integration of human input and machine analytics makes it simpler to locate faults more quickly, more accurately, and more succinctly. The framework allows operators to receive warnings that have knowledge of the context and contain valuable guidance to enable them to make decisions and be more mindful of what is happening.
5.3 FUTURE DIRECTIONS
The primary objectives of future studies will be to make the A-HIL-FD framework more scalable and resilient. With probabilistic modeling and sophisticated sensor fusion methods, it is possible for the system to work even under noisy or incomplete data. Redesigning resilience to be more resilient can be achieved by optimizing reinforcement learning algorithms to react faster to exceptional or infrequent failure occurrences. With integration into distributed computing and edge processing, it is simpler to implement in real-time in massive transmission systems. By investigating adaptive visualization and interface personalization, operators can further reduce their cognitive load, which will make decision-making safer and more efficient. Lastly, the operational flexibility and generalizability of the framework will be tested by extending it to microgrids and networks incorporating renewable energy sources.
REFERENCES
He, C., Mahfouf, M., & Torres-Salomao, L. A. (2020). An adaptive general type-2 fuzzy logic approach for psychophysiological state modeling in real-time human–machine interfaces. IEEE Transactions on Human-Machine Systems, 51(1), 1–11.
Gordleeva, S. Y., Lobov, S. A., Grigorev, N. A., Savosenkov, A. O., Shamshin, M. O., Lukoyanov, M. V., … & Kazantsev, V. B. (2020). Real-time EEG–EMG human–machine interface-based control system for a lower-limb exoskeleton. IEEE Access, 8, 84070–84081.
Tao, Y., Qiu, J., Lai, S., Zhang, X., Wang, Y., & Wang, G. (2021). A human–machine reinforcement learning method for cooperative energy management. IEEE Transactions on Industrial Informatics, 18(5), 2974–2985.
Peters, I., & Kamrul, G. (2025). Applications AI-driven solar energy management system for smart grids using predictive analytics and adaptive control. Journal of Quantum Nano-Green Environmental Systems, 1(1), 14–24. https://doi.org/10.70023/qnges.251102
Guo, L., Lu, Z., & Yao, L. (2021). Human–machine interaction sensing technology based on hand gesture recognition: A review. IEEE Transactions on Human-Machine Systems, 51(4), 300–309.
Lavrov, E., Siryk, O., Rybalov, B., & Velykodnyi, D. (2021, October). The method of adaptive control of the process of human–machine interaction to ensure the reliability and efficiency of automated control systems. In 2021 IEEE 8th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T) (pp. 403–407). IEEE.
Wang, M. H., Lu, S. D., & Liao, R. M. (2021). Fault diagnosis for power cables based on convolutional neural network with chaotic system and discrete wavelet transform. IEEE Transactions on Power Delivery, 37(1), 582–590.
Memon, A. A., & Kauhaniemi, K. (2021). Real-time hardware-in-the-loop testing of IEC 61850 GOOSE-based logically selective adaptive protection of AC microgrid. IEEE Access, 9, 154612–154639.
Ren, Y., & Li, G. P. (2022). An interactive and adaptive learning cyber physical human system for manufacturing with a case study in worker–machine interactions. IEEE Transactions on Industrial Informatics, 18(10), 6723–6732.
Wang, C., Du, Y., Chang, Y., Guo, Z., & Huang, Y. (2023). Human–Machine Collaborative Reinforcement Learning for Power Line Flow Regulation. IEEE Transactions on Industrial Informatics, 20(4), 5087–5099.
Fan, S., Guo, J., Ma, S., Li, L., Wang, G., Xu, H., … & Zhao, Z. (2023). Framework and key technologies of human–machine hybrid-augmented intelligence system for large-scale power grid dispatching and control. CSEE Journal of Power and Energy Systems, 10(1), 1–12.
Scibilia, A., Pedrocchi, N., & Fortuna, L. (2023). Modeling nonlinear dynamics in human–machine interaction. IEEE Access, 11, 58664–58678.
Sahu, S. K., Roy, M., Dutta, S., Ghosh, D., & Mohanta, D. K. (2023). Machine learning-based adaptive fault diagnosis considering hosting capacity amendment in active distribution network. Electric Power Systems Research, 216, 109025.
Mazhar, T., Shahzad, T., Rehman, A. U., & Hamam, H. (2025). Integration of smart grid with Industry 5.0: Applications, challenges and solutions. Measurement: Energy, 5, 100031.
Jain, B., Sonwane, V., Awasti, S. P., & Gupta, K. D. (2024, October). Adaptive concept of human–machine interaction with ubiquitous networks. In 2nd International Conference on Pervasive Computing Advances and Applications (PerCAA 2024) (pp. 108–116). IET.
Jin, M., Li, Y., Xu, Y., Zhang, J., Yang, F., & Su, Y. (2024). Study on construction of power grid decision agent and human–machine collaboration dynamic optimization technology. In International Conference of Electrical, Electronic and Networked Energy Systems (pp. 137–144). Springer Nature.
Xu, C., Chen, J., Zhu, Z., Liu, M., Lan, R., Chen, X., … & Li, H. (2024). Flexible pressure sensors in human–machine interface applications. Small, 20(15), 2306655.
Hamdani, R., & Chihi, I. (2025). Integrating adaptive human–computer interaction in fault detection and diagnosis systems: A human-centric approach for Industry 5.0. In International Conference on Advanced Information Networking and Applications (pp. 444–457). Springer Nature.
Tong, Y. (2025). Enhancing user experience in human-robot interaction in manufacturing (Doctoral dissertation, Cardiff University).
Feng, J., Yu, T., Zhang, K., & Cheng, L. (2025). Integration of multi-agent systems and artificial intelligence in self-healing subway power supply systems: Advancements in fault diagnosis, isolation, and recovery. Processes, 13(4), 1144.
Zhang, Q., Xu, C., Li, J., Sun, Y., Bao, J., & Zhang, D. (2025). LLM-TSFD: An industrial time series human-in-the-loop fault diagnosis method based on a large language model. Expert Systems with Applications, 264, 125861.
Nderitu, J. H. (2023). Mental State Adaptive Interfaces as a Remedy to the Issue of Long-term Continuous Human Machine Interaction. Journal of Robotics Spectrum, 1, 078-089.
Chakraborty, S. (2025). Supporting Sense-Making and Resilience in Distributed Cognition Systems: Managing Cognitive Load and Uncertainty in Traffic Control Rooms (Doctoral dissertation, ETH Zurich).
Marot, A., Rozier, A., Dussartre, M., Crochepierre, L., & Donnot, B. (2020). Towards an ai assistant for power grid operators. arXiv preprint arXiv:2012.02026.
Okpara, K. (2025). Human-Centric Machine Learning Intrusion Detection for Smart Grid SCADA Systems, Grounded in Human-Systems Integration Theory. Available at SSRN 5295278.
Pacaux-Lemoine, M. P. (2020). HUMAN-MACHINE COOPERATION: Adaptability of shared functions between Humans and Machines-Design and evaluation aspects (Doctoral dissertation, Université Polytechnique Hauts-de-France).
Sutarto, A. P., Herlambang, M. B., Izzah, N., & Hendi, A. (2025). Optimizing Mental Workload Detection for HCI: Comparative Feature Selection and Interpretable Machine Learning. International Journal of Computing, 17(1), 1-17.
Pacaux-Lemoine, M. P., Berdal, Q., Guérin, C., Rauffet, P., Chauvin, C., & Trentesaux, D. (2022). Designing human–system cooperation in industry 4.0 with cognitive work analysis: a first evaluation. Cognition, Technology & Work, 24(1), 93-111.
Amokrane-Ferka, K., Marot, A., Meddeb, M., Dussartre, M., Crochepierre, L., Rozier, A., … & Achour, W. (2024). Framework for Human and AI-assistant bidirectional interaction applied to industrial system operations.
Marot, A., Kelly, A., Naglic, M., Barbesant, V., Cremer, J., Stefanov, A., & Viebahn, J. (2022). Perspectives on future power system control centers for energy transition. Journal of Modern Power Systems and Clean Energy, 10(2), 328-344.
Matthews, G., Panganiban, A. R., Lin, J., Long, M., & Schwing, M. (2021). Super-machines or sub-humans: Mental models and trust in intelligent autonomous systems. In Trust in Human-Robot Interaction (pp. 59-82). Academic Press.
Besigomwe, K. (2025). Human-in-the-Loop Self-Healing Systems: Integrating Human Oversight for Autonomous Failure Detection, Repair and System Optimization.
Mussi, M., Metelli, A. M., Restelli, M., Losapio, G., Bessa, R. J., Boos, D., … & Zanotti, G. (2025). Human-AI interaction in safety-critical network infrastructures. iScience, 28(9).
Srivastava, A. K. (2021). FW-HTF: Collaborative Research: Augmenting and Advancing Cognitive Performance of Control Room Operators for Power Grid Resiliency. NSF Award Number 2213359. Directorate for Computer and Information Science and Engineering, 22(2213359), 13359.
https://data.mendeley.com/datasets/3dvjgvv5bz/2

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top