Loading...
Thumbnail Image
Publication

Discourse Elements Classification in Argumentative Essays: Exploring Differences Across Writing Quality, Tasks, Prompts, and Writer Demographics

Wan, Qian
Citations
Altmetric:
Abstract

Argumentative writing is crucial in education, allowing students to communicate effectively and persuasively. However, manual essay scoring is resource-intensive, prompting the development of Automated Writing Evaluation (AWE) systems, which traditionally focus on surface-level features like spelling, grammar, mechanics, and vocabulary, often neglecting the analysis of discourse elements and structure. Recent advancements in automated discourse element classification, i.e., categorizing segments into types like claims, counterclaims, and evidence, show promise for enhancing AWE systems. Despite this, research is hindered by the lack of large-scale datasets and neglecting issues related to model generalizability, bias, and fairness. This dissertation addresses these gaps using the PERSUADE corpus, a substantial dataset of students' persuasive argumentative essays containing more than 280,000 annotated discourse elements with writer demographic information. This dissertation evaluates methods for classifying discourse elements in student essays using both traditional NLP-based models and state-of-the-art transformer architectures, addressing generalizability and fairness concerns. The findings reveal that transformer-based models (RoBERTa and DeBERTa) outperformed traditional models (SGD and Random Forest), with DeBERTa achieving the highest performance across discourse element types. While classification accuracy was primarily driven by discourse-specific features such as element length, effectiveness levels, and inter-rater agreement, demographic factors, particularly gender and socioeconomic status, influenced performance in transformer model (DeBERTa), highlighting potential fairness concerns in educational large language model applications. The dissertation offers both theoretical contributions to discourse-level classification research and practical implications for developing scalable and interpretable AWE systems for K-12 student populations. The developed models, particularly the transformer-based DeBERTa, demonstrate high classification accuracy and generalizability across writing prompts and task types, while also highlighting concerns about fairness for certain demographic subgroups. Beyond model performance, the dissertation highlights the complementary strengths of interpretable feature-based models and high-performing transformer architectures, advocating for hybrid approaches that integrate both predictive power and linguistic transparency. The findings establish a replicable and extensible framework for automated discourse classification, with potential applications in instructional feedback generation, and future real-time, end-to-end discourse analysis systems.

Comments
Description
Date
2027-07-24
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
Argumentative writing, Discourse element classification, Automated writing evaluation, Transformer models, Educational AI fairness, Model generalizability, Natural language processing
Citation
Wan, Qian. "Discourse Elements Classification in Argumentative Essays: Exploring Differences Across Writing Quality, Tasks, Prompts, and Writer Demographics." PhD diss., Georgia State University, 1905. https://doi.org/ma9b-et36.
Embargo Lift Date
2027-07-24
Embedded videos