WHAT KIND OF EDUCATIONAL RELATIONSHIP DOES A LARGE LANGUAGE MODEL CONSTRUCT? A QUALITATIVE ANALYSIS OF HUMAN ANNOTATIONS IN UNIVERSITY GUIDANCE DIALOGUES

Authors

  • Luna Lembo Pegaso Telematic University, Italy
  • Giuseppe Caggianese National Research Council (CNR)
  • Stefania Morsanuto Pegaso net University
  • Agnese Augello National Research Council (CNR)
  • Davide Perrotta Pegaso net University
  • Pietro Neroni National Research Council (CNR)
  • Luca Sabatucci National Research Council (CNR)
  • Francesco Peluso Cassese Pegaso net University

DOI:

https://doi.org/10.32043/gsd.v10i4.1939

Keywords:

University guidance, LLMs, Educational relationship

Abstract

The adoption of Large Language Models (LLMs) in university guidance is reshaping how students explore academic pathways, raising a key pedagogical question: what kind of educational relationship does conversational AI construct during guidance dialogues? Developed within the ELOQUENCE project (HORIZON-CL4-2023-HUMAN-01-CNECT), this study extends the framework of synthetic personas (Casoria et al., 2025) and synthetic dialogues for bias elicitation (Augello et al., 2025), shifting the focus from algorithmic fairness to educational relationship quality. Through a thematic analysis of human annotations on university guidance dialogues, six pedagogical dimensions emerged: pedagogical listening, recognition of student uniqueness, continuity, agency, reflective scaffolding, and educational reciprocity. Findings show that, although LLMs produce coherent interactions, they remain limited in supporting authentic listening, personalization, and students' autonomous decision-making, providing a pedagogical framework for evaluating AI-supported guidance systems.

Downloads

Published

2026-10-30