This contribution describes the design and evaluation of SAM, a GPT3-based chatbot trained to hold conversations using Socratic dialogue rules. The system was tested by twenty users and two types of analysis were carried out, one from the user’s point of view, the other from a technical point of view, to verify compliance with the principles of the Socratic dialogue. The results show a satisfactory performance concerning the non-violation of the basic tenets of the approach and a highly positive user experience, especially regarding the perceived understanding abilities and competence of SAM and its effect on provoking reflection.
SOCRATIC ARTIFICIAL MIND (SAM); Lessons from the evaluation of a GPT-3 based chatbot for Socratic dialogue
Calcagno S.;Giordano D.
2023-01-01
Abstract
This contribution describes the design and evaluation of SAM, a GPT3-based chatbot trained to hold conversations using Socratic dialogue rules. The system was tested by twenty users and two types of analysis were carried out, one from the user’s point of view, the other from a technical point of view, to verify compliance with the principles of the Socratic dialogue. The results show a satisfactory performance concerning the non-violation of the basic tenets of the approach and a highly positive user experience, especially regarding the perceived understanding abilities and competence of SAM and its effect on provoking reflection.File in questo prodotto:
Non ci sono file associati a questo prodotto.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
