ORCID
- Nouman Ashraf: 0000-0002-0164-8031
Abstract
Mental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literature for the detection of mental illness. However, current systems exhibit suboptimal performance primarily due to their reliance on traditional embedding techniques and generic language models to generate text embeddings. To address this limitation, there is a requirement for domain-specific pretrained language models that comprehensively understand the context found in posts of person with a psychiatric disability patients. Posts from individuals with mental illness often contain metaphorical expressions, posing a challenge for existing models in understanding such figurative language. In this study, we propose a hybrid transformer architecture, comprising MentalBERT and MelBERT pretrained language models, cascaded with CNN models to generate and concatenate deep features. MentalBERT is pretrained on an extensive corpus of text data specifically related to the mental health domain, while MelBERT is trained on a large corpus of metaphorical data for improved understanding of metaphorical expressions. The results reveal outstanding performance of the proposed architecture with an overall accuracy of 92% and an F1-score of 92%, surpassing state-of-the-art models in comparison. This study underscores the necessity for further research in this field and illustrates the potential of advanced technologies to address mental health issues in contemporary society.
Keywords
Convolutional neural network, deep learning, melBERT, mental illness, MentalBERT, social media, transformer
DOI Link
Publication Date
2025-01-01
Publication Title
IEEE Access
Volume
13
First Page
12148
Last Page
12167
Deposit Date
2026-10-22
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Additional Links
Recommended Citation
Karamat, A., Imran, M., Yaseen, M., Bukhsh, R., Aslam, S., & Ashraf, N. (2025) 'A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text', IEEE Access, 13, pp. 12148-12167. Available at: 10.1109/ACCESS.2024.3519308