Abstract:

While rapid developments within transformer-based language models significantly improved emotion detection in NLP applications, these models resulted in high-dimensional embeddings that were computationally very expensive and hard to interpret. With emotion-aware systems expanding across social platforms into conversational AI, there was an urgent need to develop representation learning techniques that could be efficient, transparent, and scalable. Generally, the dense contextual embeddings introduced redundancy, hence motivating the interest in exploring more selective and interpretable feature representations.  Most existing methods focus either on attention-head pruning or token-level reduction and do not consider any mechanism for fine-grained selection at the embedding dimension level. Previous systems therefore could not meet the optimal tradeoff between efficiency, accuracy, and interpretability. This work is designed to develop and evaluate an embedding-level feature selection framework to enhance the efficiency and interpretability of transformer-based models of emotion detection.  The proposed framework applied a multi-criteria scoring method to incorporate gradient-based saliency, mutual information, variance filtering, and semantic alignment into ranking and pruning redundant embedding dimensions. Experiments were performed on the GoEmotions, MELD, and DailyDialog datasets by utilizing BERT-base embeddings and training light-weight MLP and BiLSTM classifiers on the resultant reduced feature sets. Performance comparisons against full-embedding baselines and traditional compression methods were performed under consistent settings. The approach was further validated through ablation studies and semantic projection analyses in order to assess interpretability gains.  This embedding compression, with the removal of 40-70% of dimensions, allowed the framework to realize 96-98% of the baseline performance in Macro-F1. This was all due to increased efficiency because of reduced parameters and, hence, inference time, and minority-class F1 scores improved by 2-6% because of the removal of noisy dimensions. Alignment of the selected embedding channels with psycholinguistic features was also better, hence increasing model interpretability. These results positioned embedding-level feature selection as a feasible approach toward the improvement of both efficiency and interpretability for transformer models in resource-constrained emotion detection systems. The approach in this work is scalable, interpretable, and privacy-aware toward deploying emotion-aware NLP applications in real-world conversational intelligence, mental health analytics, and social media monitoring.