Jiyong Wang

China University of Petroleum-Beijing at Karamay, Karamay, China

wang200212086666@163.com

Abstract—In this paper, we explore the application of a deep learning approach integrated with an attention mechanism for text classification, aiming to enhance the performance and robustness of complex text classification tasks. Addressing the limitations of traditional text classification methods in capturing contextual dependencies, handling long texts, and dealing with semantic ambiguity, we propose an innovative model that leverages deep learning architectures combined with a self-attention mechanism to improve feature extraction by dynamically focusing on key information within the text. The model employs word embeddings and a hierarchical network structure, with a multi-head attention mechanism at its core. Additionally, the model is optimized for specific task objectives to achieve efficient multi-class text classification. Experimental results on multiple publicly available datasets show that the proposed method achieves superior performance compared to existing state-of-the-art approaches, particularly in terms of accuracy and F1 score. This is especially evident when handling long and complex text data, where the model demonstrates enhanced capability in capturing relevant features and improving classification accuracy. Furthermore, a visual analysis of the attention weight distribution is conducted, offering insights into the model’s text feature extraction mechanism. These findings provide valuable theoretical contributions and practical guidance for advancing text classification techniques in the field of natural language processing.

Keywords—deep learning architecture, self-attention mechanism, word embedding, hierarchical network structure

                                                                                                  I.     Introduction

Text classification is one of the core tasks, which is widely used in sentiment analysis, subject recognition, spam filtering and other practical scenarios. However, the traditional deep learning model still has some limitations in capturing complex text structures and semantic dependencies, especially when dealing with long text, highly dependent context information, and multi-class tasks, the model’s representation ability may be weakened.

This paper proposes a new text classification method combined with deep learning. By assigning dynamic weights to different input information, attention mechanism can effectively highlight the contribution of key information and restrain the interference of redundant information, so as to improve the expression ability of text features. In this study, a text classification model adapted to different task requirements is designed by combining Multi-Head Attention mechanism and deep neural network architecture. The model converts discrete text data into a high-dimensional vector space representation by Word Embedding [1-3], and further captures the semantic correlation within the text.

In addition, the model introduces an adaptive adjustment strategy to optimize the objective function to improve the robustness of multi-class classification tasks. Through comparative experiments on public data sets, the significant advantages of the proposed method in accuracy, accuracy, recall and F1 scores are verified. Through the visual analysis of attention weight, the internal characteristics and decision-making process of the model are deeply discussed, which provides a new technical path and theoretical support for complex text classification tasks.

The second part introduces the relevant research background and the shortcomings of existing methods; The third part elaborates the model structure and key techniques proposed in this paper. The fourth part shows the experimental design and result analysis; Finally, the research contribution is summarized and the future research direction is discussed.

                                                                                                II.    Related work

Text classification is one of the core tasks of NLP, the goal of which is to classify text into predefined categories based on its content. Traditional text classification methods mainly rely on models based on statistical learning, such as Naive Bayes, Support Vector machines (SVM), and Logistic Regression. These methods are usually represented by features such as Bag of Words (BoW) and word frequency-inverse document frequency (TF-IDF). However, such methods are unable to capture the semantic information and context dependence of text effectively, especially when dealing with long text or complex text structure.

Models such as CNN and RNN [4-6] have greatly enhanced text classification performance by automatically capturing hierarchical features. CNNS are excellent at extracting local features, while RNNS are suitable for dealing with time-dependent texts through their sequential structure. However, these models still face some challenges, such as the inability of CNNS to effectively capture long-term dependencies, the problem of gradient disappearance when RNNS deal with long text, and the long training time.

Researchers introduce the Attention Mechanism, which is based on the ability of prioritizing key information in human cognition. By dynamically assigning weights, the attention mechanism enables the model to effectively focus on the most important part of the input sequence for classification tasks. The introduction of attention mechanism has significantly enhanced the ability of models to capture complex text semantics, and promoted the proposal of several innovative models.

The original single-headed attention mechanism highlighted the importance of key parts by assigning weights to each element in the input sequence. For example, Bahdanau et al. introduced an RNN-based attention mechanism in machine translation tasks that enables the model to dynamically select context information, thus alleviating the gradient disappearance problem of traditional RNNS when dealing with long text. However, the modeling ability of single-headed attention mechanism is limited, and its computational attention distribution is prone to information loss in complex scenes, and it is difficult to fully capture diversified semantic associations.

To overcome the above problems, Multi-Head Attention mechanism [7-9] has emerged and become a core component of Transformer[10-12] model. By introducing multiple independent attention heads, the multi-head attention mechanism enables the model to capture rich semantic information from different representation Spaces in parallel. This mechanism shows significant advantages in long text processing, context modeling, and semantic feature extraction. The model represented by Transformer not only completely abandons the sequence-dependent restriction of RNN, but also realizes the global modeling of text features through self-attention operation, providing a more efficient solution for text classification tasks.

The Self-Attention mechanism further expands the application depth of attention mechanism. By calculating the global correlation between each word in the sequence and other words, the mechanism models the long-term dependency relationship in the text from the global perspective, making the model have remarkable ability in capturing the context consistency and multi-granularity semantic features of the text. Pre-trained models based on self-attention mechanisms [13-15], such as BERT and RoBERTa, have achieved significant performance gains on a variety of natural language processing tasks through large-scale unsupervised pre-training and task-specific fine-tuning. This kind of model has become the benchmark method in text classification because of its excellent semantic understanding ability and task transfer performance.

For example, by embedding attention mechanisms into Convolutional Neural Networks (CNNS), the local feature extraction advantages of CNNS can complement the global semantic modeling capabilities of attention mechanisms. When it is introduced into Graph Neural Networks (GNN), the complex graph structure relationship between texts can be captured. In addition, Hierarchical Attention Networks (HAN) further improve the processing of long documents by modeling the distribution of attention in sentences and documents in layers.

Although the development of attention mechanism has significantly promoted the research progress of text classification, there are still some technical difficulties in its practical application. For example, the computational complexity of multi-head attention mechanism increases square with the increase of input sequence length, which limits the real-time processing capability of the model. However, in the multi-task scenario, how to design a unified attention distribution strategy to meet the needs of different tasks is still an urgent research problem. This paper proposes a new model combining multi-head attention mechanism and deep learning, which not only improves the performance of text classification, but also focuses on optimizing computational complexity and resource consumption, so as to provide a new solution for further research and practical application of text classification.

                                                                                          III.   Theoretical basis

A.    Multi-Head Attention

Attention mechanism is a technique that improves model performance by focusing on key parts of input information. In the study of hydrological information processing and flood prediction, the introduction of attention mechanism also has significant potential. The following is the specific application and improvement plan of multi-head attention mechanism combined with attention mechanism in hydrological information processing:

The original sequence of obtained information  is first transformed linearly. Respectively mapped to query vector sequence , key vector sequence  and value vector sequence . The mapping relationship can be expressed as:

Where , ,  are linear transformation matrices of randomly initialized queries, keys and values. This step aims to project features on the original data to provide a uniform semantic space for subsequent attention calculations.

Based on the final query vector , the dot product is computed one by one with the sequence of key vectors  to obtain an unnormalized attention score:

Where  is the dimension of the key vector, which is used to scale the dot product result to avoid excessive range of values. This step captures key parts of the data with strong correlations by dynamically assigning attention weights.

The unnormalized attention scores were normalized by using Softmax function, and the weight vector was obtained :

The normalized weights reflect the relative importance of each value vector in the output.

The normalized weights  as the weighting matrix, Weighted summation with the sequence of value vectors :

Among them,  for weighting matrix, the output sequence  after linear layer formed finally predicted.

In traditional since the attention mechanism, all the query vector  calculate the dot product. However, for the n-to-one or n-to-m tasks in hydrologic information processing, the output sequence is usually later than the input sequence and has obvious time unidirectivity. Under this background, the early query vector  have less effect on the final output, the amount of information is usually lower than . Therefore, we improve the dot product operation to calculate only the final query vector  and key vector sequence, so as to better conform to the physical law and time dependence in hydrological information.

The multi-head attention mechanism performs the above attention computation process in parallel by dividing the input data into multiple parts, each corresponding to an independent attention head. Finally, the output of each attention head is spliced and fed into the linear layer to produce the final result. This mechanism can capture rich information correlation from multiple semantic Spaces, and greatly improve the expressiveness and prediction performance of the model. As shown in Figure 1, the key adjustment made in this study compared to the traditional self-attention mechanism is to use the last query item to calculate the attention score instead of using the entire query sequence. This change is based on the following considerations: Traditional self-attention mechanisms are designed to capture long-distance dependencies in text sequences, where each query item has a potential correlation to the entire key sequence. However, in text information processing tasks, due to the chronological and causal nature of the input data, the later data often contain more predictive value, especially the data at the last time point, which has a decisive impact on the prediction of future hydrological conditions. Therefore, selection as the basis for calculating attention scores can more accurately capture important associations between data, thereby improving prediction accuracy. Furthermore, the multi-head attention mechanism allows the model to focus on different features of the input data from multiple perspectives at the same time. In terms of implementation, the input data can be divided into multiple subsets, each subset is processed independently according to the above steps, and finally the output of each head is combined to form a more rich and comprehensive data representation.

• Self-attention mechanism frame diagram.

B.    Recurrent Neural Network

RNN, as a deep learning model, is designed for processing continuous sequential data, especially for time-dependent input data such as speech signals. The core idea is to expand the speech signal according to the time dimension and input it into the network moment by moment. At the same time, the time series information is integrated into the calculation of the model through the state transmission mechanism of the hidden layer, so as to capture the dynamic characteristics of the sequence data.

The basic structure of an RNN is shown in the figure 2. For any moment t, input layer, hidden layer and output layer respectively to ,   and . The model is calculated by the following weight matrices:

 : Weight matrix from input layer to hidden layer;

 : The weight matrix from the previous hidden layer state to the current hidden layer state;

 : Hides the weight matrix from the output layer.

The core calculation formula is as follows:

As can be seen from the above formula, the value of the hidden layer state   is determined by the input  at the current moment and the hidden layer state at the previous moment. At the same time, the weight matrix  makes the state of the hidden layer pass over the time dimension, thus introducing the context of the time series into the calculation process. This design enables RNN to capture dynamic dependencies of time series data effectively and has significant advantages for processing time series data such as speech signals.

Because RNNS are able to incorporate state correlations from before and after moments into their calculations, they perform well when working with time series data, such as voice, text, or sensor data. However, when the sequence length is long, RNNS are prone to the problems of gradient disappearance and gradient explosion. Gradient disappearance: In the process of backpropagation, the gradient at the distant time gradually decreases to close to zero, which makes it difficult for the network to capture the distant dependence; Gradient explosion: Gradient values increase rapidly in backpropagation, which may lead to unstable parameter updates and even make the model unable to converge.

The root cause of the above problem is that the hidden layer state  will gradually weaken or strengthen with the increase of the propagation distance during the transmission process on the time axis. RNNS have been widely used in speech recognition, natural language processing (such as machine translation, text generation), and time series prediction. For example, in speech signal processing, RNNS are able to dynamically adjust the weights of the model based on continuous changes in the speech signal, thus capturing speech features more accurately. In addition, RNN variants incorporating attention mechanisms further improve the model’s ability to deal with long sequence dependencies.

• RNN structure diagram.

               IV.   Text classification algorithm based on multi-head attention mechanism -RNN

This paper presents a text classification algorithm that combines the Multi-Head Attention Mechanism with a Recurrent Neural Network (RNN), referred to as MA-RNN (Multi-Head Attention RNN). The algorithm aims to improve the accuracy of text classification, especially when dealing with texts containing long sequences and complex context dependencies, and can fully mine the global semantics and timing information of texts. The working principle of MA-RNN algorithm can be divided into the following steps:

1. Word embedding and attention mechanism: First, the input text data is converted by the word embedding layer into a sequence of word vectors , where each word vector is a representation of the corresponding word. Then, through the multi-head attention mechanism, the Query, Key, and Value representations of the text are mapped to different subspaces. For each attentional head, a sequence of questions, answers, and values is first obtained by linear transformations:

Among them, ,  and  respectively for queries, key and the value of the weight matrix.

2. Calculate the attention score: By calculating the dot product of the query vector and all key vectors , get the correlation degree of each word with the current word, and then get the attention score. Attention scores are calculated using the dot product scaling formula as follows:

is key vector dimensions,  said the first i word for the first n the word attention points.

3. Weight normalization: Attention score is normalized by softmax function, and the final attention weight is obtained:

These weights represent the importance of each word at the current moment, and these weights are then applied to the sequence of values , yielding the output of the weighted summation.

4. RNN modeling timing dependencies: The text representation processed by the attention mechanism is passed into the RNN module, which is used to capture the timing dependencies in the text. Suppose the hidden state of RNN is , its calculation formula is:

One, the  said the first t time input,  is the hidden state of the moment  is the weight matrix.

5. Classification output: The final hidden state of the RNN is used as a representation of the text, passed to the full connection layer (usually softmax layer) for classification:

One, the  and  is a classification layer weights and bias,   said the final classification result.

By combining multi-head attention mechanism with RNN, MA-RNN can effectively capture global information (through multi-head attention mechanism) while processing text classification tasks, and can also capture timing dependencies through RNN, thus improving the classification accuracy of the model. The algorithm is especially suitable for processing long text and complex semantic text classification tasks, such as sentiment analysis, news classification, text classification in question answering system and so on.

                                                                                     V.    Experimental analysis

A.    Data set

The text classification dataset used in this study covers multiple domains and types of text data to evaluate the performance of multi-head attention-based and recurrent neural network (MA-RNN) algorithms in different text classification tasks. The dataset is derived from public text classification databases, such as IMDb Sentiment analysis dataset, 20 Newsgroups dataset, and Amazon product review dataset, and contains rich text content and label information, which is suitable for a variety of text classification tasks. The size of the dataset ranges from about 100,000 articles, and the text length ranges from dozens to hundreds of words, covering multiple sub-tasks such as sentiment analysis, news classification and product reviews, including 5 types of sentiment analysis tasks, 20 types of news classification tasks and 10 types of product reviews tasks. Each text data is attached with a corresponding category label, and the category distribution is relatively balanced, ensuring the fairness of model training and evaluation.

The text content comes from news articles, movie reviews, social media posts, and product reviews on e-commerce platforms, and has a high diversity and complexity. The distribution of class labels in the data set is roughly uniform, which avoids the influence of class imbalance on model performance. To ensure data consistency and validity, all text data is pre-processed in a standardized manner, including stop word removal, punctuation cleaning, stem extraction, and word embedding transformation. These pre-processed text data were used to train and test the MA-RNN model in handling complex semantic understanding and time-dependent modeling tasks.

B.    Evaluation indicators

In order to evaluate the performance of MA-RNN algorithms in text classification tasks, this study selected a variety of common and reliable evaluation indicators. First, Accuracy is used to measure the proportion of samples correctly classified by the model, and is the most intuitive performance evaluation standard. Secondly, Precision, Recall and F1 score are used to comprehensively evaluate the classification ability of the model in various categories, especially in the case of unbalanced categories. F1 score, as a harmonic average of precision and recall, can provide more comprehensive performance feedback.

In addition, to further test the balanced performance of the model across different categories, the accuracy, recall, and F1 scores were weighted using Macro and Micro averages. Macro average is used to calculate the evaluation index of each category and take its average value, which is suitable for dealing with the imbalance between categories. Micro-average is to calculate a total evaluation index after the summary of all categories of samples, which can reflect the overall classification performance.

C.   Experimental setup

In the experimental part, in order to evaluate the performance of multi-head attention-based and recurrent neural network (MA-RNN) algorithms in the text classification task, we carried out detailed parameter Settings. First, the word embedding layer uses a pre-trained GloVe word vector (50 dimensions) to better capture semantic relationships between words in the text. For the maximum length of the input text, we set it to 500, more than the part is truncated, less than the part is filled, to ensure the standardization.

In the model design, the multi-head attention mechanism is configured with 8 heads to capture diverse contextual information from multiple subspaces. Each attentional head has a dimension of 64, ensuring variety and richness when capturing text information. The cyclic neural network adopts the Bi-LSTM (Bi-LSTM) structure, and the dimension of the hidden layer is set to 128, so that the model can capture the information before and after the text at the same time.

For the optimization algorithm, we selected the Adam optimizer, whose initial learning rate was 0.001, and used the learning rate attenuation strategy to gradually reduce the learning rate. The regularization of the model adopts Dropout method and the drop rate is 0.5 to improve the generalization ability of the model.

D.   Experimental result

Table 1 presents the experimental results of text classification tasks based on multi-head attention mechanism and recurrent neural network (MA-RNN) model compared with traditional support vector machine (SVM) and Bi-LSTM. In the experiment, the performance of each model was evaluated by Macro Average and Micro Average, and the performance of MA-RNN model was better than that of other comparison models on the two evaluation indexes, the macro average was 95.33, and the micro average was 96.38. This shows that MA-RNN shows strong ability in the overall classification task, especially in the good balance between different categories. The macro average evaluates the independent performance of each category, and the high score of MA-RNN in this respect shows its ability to accurately identify each category, especially in the case of unbalanced categories, to maintain high classification accuracy. Micro-average is a weighted average of all samples. The excellent performance of MA-RNN further proves its stability and efficiency in the overall classification task, and it can effectively process different types of text data.

The traditional SVM model still shows relatively good performance in the text classification task, with an average of 90.15 macros and 91.09 micros. SVM is a classical machine learning algorithm, which can achieve good classification effect by maximizing classification interval. Although it has slightly lower scores than MA-RNN on macro and micro-average, its performance is still relatively robust, especially when dealing with small data sets, SVM often provides more accurate classification results. However, SVM is still underperforming compared to deep learning models in processing long text and capturing complex contextual information, especially when the data volume is large and the differences between categories are complex, SVM performance may be limited.

Bidirectional LSTM (Bi-LSTM) can consider the context information at the same time when processing text data, which shows a strong ability of time series modeling. In the experimental results, the macro average of Bi-LSTM is 89.61, and the micro average is 90.87, which is slightly lower than MA-RNN and SVM, but compared with traditional RNN, its bidirectional structure enables the model to capture richer text information, especially in the processing of long text or tasks with complex context relationships, and can provide better performance. However, due to the high computational overhead of Bi-LSTM on long time series and the possible problems of disappearing or exploding gradients, its performance in some tasks is not as good as MA-RNN model, especially when dealing with long text, MA-RNN effectively alleviates these problems by introducing multi-head attention mechanism.

The introduction of multi-head attention mechanism enables MA-RNN to focus on different context information and important features adaptively in text classification tasks, thus improving the model’s ability to distinguish different categories. Compared with traditional SVM and Bi-LSTM, MA-RNN has obvious advantages in dealing with long text and complex context, especially in macro average and micro average, which show higher precision.

In general, MA-RNN far outperforms SVM and Bi-LSTM in text classification tasks, especially in macro average and micro-average, showing its advantages in capturing complex features, processing long text, and unbalanced categories. The excellent performance of this model provides strong empirical support for the combination of multi-head attention mechanism and RNN, and proves its effectiveness and application potential in natural language processing tasks.

• Comparative Experimental Outcomes Across Various Baseline Conditions

Model	Macro Average	Micro Average
MA-RNN	95.33	96.38
SVM	90.15	91.09
Bi-LSTM	89.61	90.87

E.    Ablation experiment

Table 2 shows the results obtained by the MA-RNN model after the Ablation Study. By removing a key component of the model, the ablation experiment separately evaluated two components of the model, the multi-head attention mechanism (MA) and the recurrent neural network (RNNS), and compared their effects on final performance. The experimental results were evaluated by Macro Average and Micro Average, and the specific analysis was as follows:

The baseline results of the MA-RNN model in the ablation experiment showed that the model scored 95.33 and 96.38 on the macro average and micro average, respectively, before removing any of the components. This result represents the optimal performance of MA-RNN model when it includes all components, namely multi-head attention mechanism and recurrent neural network, indicating that the model has strong comprehensive ability in processing text classification tasks, especially in terms of balance between categories and overall classification accuracy.

In the experiment after removing the multi-head attention mechanism (MA-RNN-w/o-MA), the macro average of the model dropped to 92.04 and the micro-average to 92.49. Compared with the complete model MA-RNN, this shows that the model’s performance deteriorates after the attention mechanism is removed, especially in the macro average (that is, the class-independent performance). When this mechanism is removed, the model loses the important ability to allocate attention adaptively, resulting in it being unable to fully mine important information in the text data, which affects the overall classification accuracy. Nevertheless, the reduction in the micro-average was relatively small (only about 3.89), which may be related to the fact that the model still captures the time-dependent relationship of the sequence data through the recurrent neural network, but the lack of adaptive focusing on key information led to a decrease in overall performance.

After removing the recurrent neural network (RNN) component (MA-RNN-w/o-RNN), the macro average of the model is 90.71 and the micro-average is 90.36. The core advantage of RNNS is their ability to capture time-dependent and long-distance information in input sequence data, which is critical for context understanding in text classification tasks. When RNNS are removed, the model loses the ability to model sequence relationships in the input text, resulting in performance degradation.

It can be seen from the results of ablation experiments that both the multi-head attention mechanism and the recurrent neural network of MA-RNN model play a key role in the final performance of the model. Removing any of the components will lead to a significant decline in model performance, especially the macro average performance of the model after removing the RNN part, proving the indispensability of RNN in capturing temporal relationships and context information in text. On the other hand, after removing the multi-head attention mechanism, although there is a decrease in performance, the decrease is relatively small, suggesting that the role of RNNS in the model is more fundamental and central.

• Ablation results

Model	Macro Average	Micro Average
MA-RNN	95.33	96.38
MA-RNN-w/o-MA	92.04	92.49
MA-RNN-w/o-RNN	90.71	90.36

                                                                                                  VI.   Conclusion

This paper introduces a text classification algorithm, MA-RNN, which combines the multi-head attention mechanism and recurrent neural network, and demonstrates its effectiveness through extensive experiments. The results reveal that the MA-RNN model outperforms traditional methods, such as support vector machine and Bi-LSTM, in both macro and micro-average metrics, highlighting its significant advantages in text classification tasks. The ablation experiment further analyzed the effects of multi-head attention mechanism and recurrent neural network on the performance of the model, and the results showed that the combined effect of the two significantly improved the performance of the model. In general, by integrating the advantages of multi-head attention mechanism and RNN, MA-RNN model can better deal with key information and long-distance dependence in text, and shows high classification accuracy and robustness, which provides a new and effective method for text classification tasks.

References

• Zamani, Hamed, and W. Bruce Croft. “Relevance-based word embedding.” Proceedings of the 40th international acm sigir conference on research and development in information retrieval. 2017.
• Jiao, Qilu, and Shunyao Zhang. “A brief survey of word embedding and its recent development.” 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). Vol. 5. IEEE, 2021.
• Ethayarajh, Kawin, David Duvenaud, and Graeme Hirst. “Understanding undesirable word embedding associations.” arXiv preprint arXiv:1908.06361 (2019).
• Bullinaria, John A. “Recurrent neural networks.” Neural Computation: Lecture 12.1 (2013).
• Caterini, Anthony L., et al. “Recurrent neural networks.” Deep neural networks in a mathematical framework (2018): 59-79.
• Grossberg, Stephen. “Recurrent neural networks.” Scholarpedia 8.2 (2013): 1888.
• Li, Jian, et al. “On the diversity of multi-head attention.” Neurocomputing 454 (2021): 14-24.
• India, Miquel, Pooyan Safari, and Javier Hernando. “Self multi-head attention for speaker recognition.” arXiv preprint arXiv:1906.09890 (2019).
• Deora, Puneesh, et al. “On the optimization and generalization of multi-head attention.” arXiv preprint arXiv:2310.12680 (2023).
• Karpov, Pavel, Guillaume Godin, and Igor V. Tetko. “A transformer model for retrosynthesis.” International Conference on Artificial Neural Networks. Cham: Springer International Publishing, 2019.
• Martinez, Juan A., and Bruce A. Mork. “Transformer modeling for low-and mid-frequency transients-a review.” IEEE Transactions on Power delivery 20.2 (2005): 1625-1632.
• Guerra, Francisco das Chagas Fernandes, and Wellington Santos Mota. “Current transformer model.” IEEE Transactions on Power Delivery 22.1 (2006): 187-194.
• Delobelle, Pieter, Thomas Winters, and Bettina Berendt. “Robbert: a dutch roberta-based language model.” arXiv preprint arXiv:2001.06286 (2020).
• Tan, Kian Long, et al. “RoBERTa-LSTM: a hybrid model for sentiment analysis with transformer and recurrent neural network.” IEEE Access 10 (2022): 21517-21525.
• Briskilal, J., and C. N. Subalalitha. “An ensemble model for classifying idioms and literal texts using BERT and RoBERTa.” Information Processing & Management 59.1 (2022): 102756.