Unveiling Personality Traits through Social Media Language Analysis: A Novel Approach using Language Models

Abstract: Personality prediction from text data, particularly from social media posts, has gained significant attention due to its wide- ranging applications in various fields such as psychology, marketing, and personalized recommendation systems. This study presents a machine learning approach for predicting personality types based on text data extracted from social media posts, focusing on Twitter. The study employs a state-of-the-art natural language processing (NLP) technique, namely BERT (Bidirectional Encoder Representations from Transformers), to encode and understand the textual content. BERT is a transformer-based model known for its effectiveness in capturing contextual information from text data. The Twitter API is utilized to retrieve a user's recent tweets, which serve as input for the personality prediction model. The preprocessing pipeline involves text cleaning steps to remove noise such as special characters, URLs, and punctuation marks....