Antonia Tolzin

M.Sc. Antonia Tolzin

Wissenschaftliche Mitarbeiterin

Telefon: +49 561 804-6437
E-Mail: antonia.tolzin[at]uni-kassel[dot]de

Website: Antonia Tolzin

Standort: Pfannkuchstraße 1
34121 Kassel

Raum: ITeG, Raum 1180, 1. Etage

Sprechstunde: Mi, 13:00-14:00 Anmeldung per E-Mail erforderlich. Weitere Termine nach Vereinbarung.

Lebenslauf

Ausbildung

(10/2018 – 02/2022) Studium der Psychologie (Master of Science) an der Universität Kassel

Thema der Masterarbeit: „The Effectiveness of Smartphone Apps for mental Health Promotion and Prevention: Systematic Review and Meta-Analysis”

(09/2017 – 03/2018) Erasmus Semester im Studiengang Psychologie an der Alpen-Adria Universität Klagenfurt in Österreich

(10/2014 – 09/2018) Studium der Lehr-, Lern- und Trainingspsychologie (Bachelor of Arts) an der Universität Erfurt

Thema der Bachelorarbeit: „Schluck- oder Spuckrichtung? - Eine Untersuchung zum Zusammenhang zwischen Artikulationsrichtung und Emotionen sowie Objektgröße bei der aktiven Sprachproduktion“

Arbeitserfahrung

(10/2021 – 03/2022) Tutorin für „Multivariate Statistische Verfahren“ im Masterstudiengang Psychologie an der Universität Kassel

(10/2021 – 03/2022) Tutorin für „Datenanalyse mit R“ im Masterstudiengang Psychologie an der Universität Kassel

(02/2021 – 01/2022) Studentische Hilfskraft in der Wirtschaftsinformatik an der Universität Kassel

(09/2020 – 12/2020) Forschungspraktikum am Zentralinstitut für Seelische Gesundheit Mannheim in der Abteilung Public Mental Health

(02/2019 – 09/2020) Studentische Hilfskraft in der Pädagogischen Psychologie an der Universität Kassel

(07/2018 – 08/2018) Praktikum in der Psychosomatischen Tagesklinik des Katholisches Krankenhaus St. Johann Nepomuk Erfurt

(08/2016 – 09/2016) Praktikum bei der Akademie Schweiger & Schmitt in Erfurt

(10/2015 – 09/2017) Studentische Hilfskraft in der Sozial-, Organisations- und Wirtschaftspsychologie an der Universität Erfurt

(10/2015 – 09/2017) Studentische Hilfskraft in der Entwicklungspsychologie an der Universität Erfurt

Forschungsinteressen

KI-gestütztes menschliches Lernen
Educational Technology
Interaktion und Gestaltung von smarten persönlichen Assistenten

Publikationen

2024[ to top ]

Tolzin, A.; Knoth, N.; & Janson, A. (2024): Worked Examples to Facilitate the Development of Prompt Engineering Skills. In: Thirty-Second European Conference on Information Systems (ECIS 2024). Paphos, Cyprus.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@inproceedings{ls_leimeister, abstract = {This paper explores the evolving field of prompt engineering in Artificial Intelligence (AI), with a focus on Large Language Models (LLMs). As LLMs exhibit remarkable potential in various educational domains, their effective use requires adept prompt engineering skills. We introduce a skill-based approach to prompt engineering and explicitly investigate the impact of using worked examples to facilitate prompt engineering skills among students interacting with LLMs. We propose hypotheses linking prompt engineering, worked examples, and perceived anthropomorphism to the quality of LLM output. Our initial findings support the critical relationship between proficient prompt engineering and the resulting output quality of LLMs. Subsequent phases will further explore the role of worked examples in prompt engineering, aiming to provide practical recommendations for educational improvement and industry application. Additionally, this research aims to shed light on the responsible utilization of LLMs in education and contribute insights to educational practice, research, and organizational development.}, address = {Paphos, Cyprus}, author = {Tolzin, Antonia and Knoth, Nils and Janson, Andreas}, booktitle = {Thirty-Second European Conference on Information Systems (ECIS 2024)}, keywords = {itegpub}, title = {Worked Examples to Facilitate the Development of Prompt Engineering Skills}, year = 2024 }
%0 Conference Paper %1 ls_leimeister %A Tolzin, Antonia %A Knoth, Nils %A Janson, Andreas %B Thirty-Second European Conference on Information Systems (ECIS 2024) %C Paphos, Cyprus %D 2024 %T Worked Examples to Facilitate the Development of Prompt Engineering Skills %U http://pubs.wi-kassel.de/wp-content/uploads/2024/06/JML_967.pdf %X This paper explores the evolving field of prompt engineering in Artificial Intelligence (AI), with a focus on Large Language Models (LLMs). As LLMs exhibit remarkable potential in various educational domains, their effective use requires adept prompt engineering skills. We introduce a skill-based approach to prompt engineering and explicitly investigate the impact of using worked examples to facilitate prompt engineering skills among students interacting with LLMs. We propose hypotheses linking prompt engineering, worked examples, and perceived anthropomorphism to the quality of LLM output. Our initial findings support the critical relationship between proficient prompt engineering and the resulting output quality of LLMs. Subsequent phases will further explore the role of worked examples in prompt engineering, aiming to provide practical recommendations for educational improvement and industry application. Additionally, this research aims to shed light on the responsible utilization of LLMs in education and contribute insights to educational practice, research, and organizational development.
Knoth, N.; Tolzin, A.; Janson, A.; & Leimeister, J. M. (2024): AI Literacy and its Implications for Prompt Engineering Strategies. In: Computers and Education: Artificial Intelligence.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@article{ls_leimeister, abstract = {Artificial intelligence technologies are rapidly advancing. As part of this development, large language models (LLMs) are increasingly being used when humans interact with systems based on artificial intelligence (AI), posing both new opportunities and challenges. When interacting with LLM-based AI system in a goal-directed manner, prompt engineering has evolved as a skill of formulating precise and well-structured instructions to elicit desired responses or information from the LLM, optimizing the effectiveness of the interaction. However, research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is lacking. This aspect is particularly important when considering the implications of LLMs in the context of higher education. In this present study, we address this issue, introduce a skill-based approach to prompt engineering, and explicitly consider the role of non-experts' AI literacy (students) in their prompt engineering skills. We also provide qualitative insights into students’ intuitive behaviors towards LLM-based AI systems. The results show that higher-quality prompt engineering skills predict the quality of LLM output, suggesting that prompt engineering is indeed a required skill for the goal-directed use of generative AI tools. In addition, the results show that certain aspects of AI literacy can play a role in higher quality prompt engineering and targeted adaptation of LLMs within education. We, therefore, argue for the integration of AI educational content into current curricula to enable a hybrid intelligent society in which students can effectively use generative AI tools such as ChatGPT.}, author = {Knoth, Nils and Tolzin, Antonia and Janson, Andreas and Leimeister, Jan Marco}, journal = {Computers and Education: Artificial Intelligence}, keywords = {itegpub}, number = {June}, title = {AI Literacy and its Implications for Prompt Engineering Strategies}, volume = 6, year = 2024 }
%0 Journal Article %1 ls_leimeister %A Knoth, Nils %A Tolzin, Antonia %A Janson, Andreas %A Leimeister, Jan Marco %D 2024 %J Computers and Education: Artificial Intelligence %N June %R 10.1016/j.caeai.2024.100225 %T AI Literacy and its Implications for Prompt Engineering Strategies %U http://pubs.wi-kassel.de/wp-content/uploads/2024/05/JML_966.pdf %V 6 %X Artificial intelligence technologies are rapidly advancing. As part of this development, large language models (LLMs) are increasingly being used when humans interact with systems based on artificial intelligence (AI), posing both new opportunities and challenges. When interacting with LLM-based AI system in a goal-directed manner, prompt engineering has evolved as a skill of formulating precise and well-structured instructions to elicit desired responses or information from the LLM, optimizing the effectiveness of the interaction. However, research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is lacking. This aspect is particularly important when considering the implications of LLMs in the context of higher education. In this present study, we address this issue, introduce a skill-based approach to prompt engineering, and explicitly consider the role of non-experts' AI literacy (students) in their prompt engineering skills. We also provide qualitative insights into students’ intuitive behaviors towards LLM-based AI systems. The results show that higher-quality prompt engineering skills predict the quality of LLM output, suggesting that prompt engineering is indeed a required skill for the goal-directed use of generative AI tools. In addition, the results show that certain aspects of AI literacy can play a role in higher quality prompt engineering and targeted adaptation of LLMs within education. We, therefore, argue for the integration of AI educational content into current curricula to enable a hybrid intelligent society in which students can effectively use generative AI tools such as ChatGPT.
Tolzin, A.; Knoth, N.; & Janson, A. (2024): Leveraging Prompting Guides as Worked Examples for Advanced Prompt Engineering Strategies. In: International Conference on Information Systems (ICIS). Bangkok, Thailand.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@inproceedings{ls_leimeister, abstract = {Artificial intelligence systems, particularly those based on large language models, are increasingly prevalent in personal and professional settings, making the skill of prompt engineering—formulating effective AI inputs—vital. This paper explores the effects and how to enhance students' prompt engineering skills based on a multi-study design. Our first study confirmed the baseline hypothesis that prompt engineering can predict AI output quality, framing it as a critical skill. We then investigated whether instructional designs (worked examples & instructions), could develop prompting skills. Using worked examples, the second study tested the effectiveness of instructional materials on enhancing prompting skills. The experiment involved 245 students who demonstrated that a brief exposure to a worked example-based prompting guide significantly improved their ability to deploy targeted prompting strategies. These findings suggest that integrating worked examples into curricula could effectively equip students with essential prompting skills, offering both theoretical and practical implications for AI education.}, address = {Bangkok, Thailand}, author = {Tolzin, Antonia and Knoth, Nils and Janson, Andreas}, booktitle = {International Conference on Information Systems (ICIS)}, keywords = {pub_nkn}, title = {Leveraging Prompting Guides as Worked Examples for Advanced Prompt Engineering Strategies}, year = 2024 }
%0 Conference Paper %1 ls_leimeister %A Tolzin, Antonia %A Knoth, Nils %A Janson, Andreas %B International Conference on Information Systems (ICIS) %C Bangkok, Thailand %D 2024 %T Leveraging Prompting Guides as Worked Examples for Advanced Prompt Engineering Strategies %U http://pubs.wi-kassel.de/wp-content/uploads/2024/10/JML_990.pdf %X Artificial intelligence systems, particularly those based on large language models, are increasingly prevalent in personal and professional settings, making the skill of prompt engineering—formulating effective AI inputs—vital. This paper explores the effects and how to enhance students' prompt engineering skills based on a multi-study design. Our first study confirmed the baseline hypothesis that prompt engineering can predict AI output quality, framing it as a critical skill. We then investigated whether instructional designs (worked examples & instructions), could develop prompting skills. Using worked examples, the second study tested the effectiveness of instructional materials on enhancing prompting skills. The experiment involved 245 students who demonstrated that a brief exposure to a worked example-based prompting guide significantly improved their ability to deploy targeted prompting strategies. These findings suggest that integrating worked examples into curricula could effectively equip students with essential prompting skills, offering both theoretical and practical implications for AI education.

2023[ to top ]

Tolzin, A.; & Janson, A. (2023): Mechanisms of Common Ground in Human-Agent Interaction: A Systematic Review of Conversational Agent Research. In: Hawaii International Conference on System Sciences (HICSS). Maui, Hawaii, USA.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@inproceedings{ls_leimeister, abstract = {Human-agent interaction is increasingly influencing our personal and work lives through the proliferation of conversational agents in various domains. As such, these agents combine intuitive natural language interactions by also delivering personalization through artificial intelligence capabilities. However, research on CAs as well as practical failures indicate that CA interaction oftentimes fails miserably. To reduce these failures, this paper introduces the concept of building common ground for more successful human-agent interactions. Based on a systematic review our analysis reveals five mechanisms for achieving common ground: (1) Embodiment, (2) Social Features, (3) Joint Action, (4) Knowledge Base, and (5) Mental Model of Conversational Agents. On this basis, we offer insights into grounding mechanisms and highlight the potentials when considering common ground in different human-agent interaction processes. Consequently, we secure further understanding and deeper insights of possible mechanisms of common ground in human-agent interaction in the future.}, address = {Maui, Hawaii, USA}, author = {Tolzin, Antonia and Janson, Andreas}, booktitle = {Hawaii International Conference on System Sciences (HICSS)}, keywords = {itegpub}, title = {Mechanisms of Common Ground in Human-Agent Interaction: A Systematic Review of Conversational Agent Research}, year = 2023 }
%0 Conference Paper %1 ls_leimeister %A Tolzin, Antonia %A Janson, Andreas %B Hawaii International Conference on System Sciences (HICSS) %C Maui, Hawaii, USA %D 2023 %T Mechanisms of Common Ground in Human-Agent Interaction: A Systematic Review of Conversational Agent Research %U https://pubs.wi-kassel.de/wp-content/uploads/2022/10/JML_890.pdf %X Human-agent interaction is increasingly influencing our personal and work lives through the proliferation of conversational agents in various domains. As such, these agents combine intuitive natural language interactions by also delivering personalization through artificial intelligence capabilities. However, research on CAs as well as practical failures indicate that CA interaction oftentimes fails miserably. To reduce these failures, this paper introduces the concept of building common ground for more successful human-agent interactions. Based on a systematic review our analysis reveals five mechanisms for achieving common ground: (1) Embodiment, (2) Social Features, (3) Joint Action, (4) Knowledge Base, and (5) Mental Model of Conversational Agents. On this basis, we offer insights into grounding mechanisms and highlight the potentials when considering common ground in different human-agent interaction processes. Consequently, we secure further understanding and deeper insights of possible mechanisms of common ground in human-agent interaction in the future.
Tolzin, A.; Körner, A.; Dickhaut, E.; Janson, A.; Rummer, R.; & Leimeister, J. M. (2023): Designing Pedagogical Conversational Agents for Achieving Common Ground. In: International Conference on Design Science Research (DESRIST). Pretoria, South Africa.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@inproceedings{ls_leimeister, abstract = {As educational organizations face difficulties in providing personalized learning material or individual learning support, pedagogical conversational agents (PCAs) promise individualized learning for students. However, the problem of conversational breakdowns of PCAs and consequently poor learning outcomes still exist. Hence, effective and grounded communication between learners and PCAs is fundamental to improving learning processes and out-comes. As understanding each other and the conversational grounding is crucial for conversations between humans and PCAs, we propose common ground theory as a foundation for designing a PCA. Conducting a design science research project, we propose theory-motivated design principles and instantiate them in a PCA. We evaluate the utility of the artifact with an experimental study in higher education to inform the subsequent design iterations. We contribute design knowledge on conversational agents in learning settings, enabling researchers and practitioners to develop PCAs based on common ground research in education and providing avenues for future research. Thereby, we can secure further understanding of learning processes based on grounding communication.}, address = {Pretoria, South Africa}, author = {Tolzin, Antonia and Körner, Anita and Dickhaut, Ernestine and Janson, Andreas and Rummer, Ralf and Leimeister, Jan Marco}, booktitle = {International Conference on Design Science Research (DESRIST)}, keywords = {itegpub}, title = {Designing Pedagogical Conversational Agents for Achieving Common Ground}, year = 2023 }
%0 Conference Paper %1 ls_leimeister %A Tolzin, Antonia %A Körner, Anita %A Dickhaut, Ernestine %A Janson, Andreas %A Rummer, Ralf %A Leimeister, Jan Marco %B International Conference on Design Science Research (DESRIST) %C Pretoria, South Africa %D 2023 %T Designing Pedagogical Conversational Agents for Achieving Common Ground %U http://pubs.wi-kassel.de/wp-content/uploads/2023/06/JML_933.pdf %X As educational organizations face difficulties in providing personalized learning material or individual learning support, pedagogical conversational agents (PCAs) promise individualized learning for students. However, the problem of conversational breakdowns of PCAs and consequently poor learning outcomes still exist. Hence, effective and grounded communication between learners and PCAs is fundamental to improving learning processes and out-comes. As understanding each other and the conversational grounding is crucial for conversations between humans and PCAs, we propose common ground theory as a foundation for designing a PCA. Conducting a design science research project, we propose theory-motivated design principles and instantiate them in a PCA. We evaluate the utility of the artifact with an experimental study in higher education to inform the subsequent design iterations. We contribute design knowledge on conversational agents in learning settings, enabling researchers and practitioners to develop PCAs based on common ground research in education and providing avenues for future research. Thereby, we can secure further understanding of learning processes based on grounding communication.
Berkle, Y.; Schmitt, L.; Tolzin, A.; Janson, A.; Wambsganss, T.; Leimeister, J. M.; & Leuchter, M. (2023): Measuring university students’ ability to recognize argument structures and fallacies. In: Frontiers in Psychology (Front. Psychol.), Ausgabe/Nummer: 1270931, Vol. 14, Erscheinungsjahr/Year: 2023. pp. 1–17.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@article{ls_leimeister, abstract = {Theory: Argumentation is crucial for all academic disciplines. Nevertheless, a lack of argumentation skills among students is evident. Two core aspects of argumentation are the recognition of argument structures (e.g., backing up claims with premises, according to the Toulmin model) and the recognition of fallacies. As both aspects may be related to content knowledge, students studying different subjects might exhibit different argumentation skills depending on whether the content is drawn from their own or from a foreign subject. Therefore, we developed an instrument to measure the recognition of both argument structures and fallacies among the groups of preservice teachers and business economics students in both their respective domains (pedagogy and economics), and a neutral domain (sustainability). For the recognition of fallacies, we distinguished between congruent and incongruent fallacies. In congruent fallacies, the two aspects of argument quality, i.e., deductive validity and inductive strength, provide converging evidence against high argument quality. In incongruent fallacies, these two aspects diverge. Based on dual process theories, we expected to observe differences in the recognition of congruent and incongruent fallacies. Aims: We investigated whether these two abilities are domain-specific and whether the recognition of fallacies depends on the congruence of two aspects of argument quality. Methods: 267 preservice teachers and 56 business economics students participated in the study. For the recognition of argument structures, participants assigned the five statements constituting one argument to the corresponding component according to the Toulmin model. For the recognition of fallacies, we created arguments and incorporated a common fallacy into some of them: formal fallacy, overgeneralization, irrelevance, or circularity. Participants rated whether the argument was cogent or not, which was followed by a brief justification. Results: Domain specificity could not be found for either of both abilities. For the recognition of fallacies, two dimensions were found: a congruent dimension (formal fallacies and overgeneralizations) and an incongruent dimension (irrelevance and circularity). Discussion: The instrument measures the recognition of both argument structures and fallacies in these two groups across domains. The recognition of fallacies differs depending on whether the deductive validity and the inductive strength of the argument are equally indicative of argument quality or not.}, author = {Berkle, Yvonne and Schmitt, Lukas and Tolzin, Antonia and Janson, Andreas and Wambsganss, Thiemo and Leimeister, Jan Marco and Leuchter, Miriam}, journal = {Frontiers in Psychology (Front. Psychol.)}, keywords = {itegpub}, number = 1270931, pages = {1-17}, title = {Measuring university students’ ability to recognize argument structures and fallacies}, volume = 14, year = 2023 }
%0 Journal Article %1 ls_leimeister %A Berkle, Yvonne %A Schmitt, Lukas %A Tolzin, Antonia %A Janson, Andreas %A Wambsganss, Thiemo %A Leimeister, Jan Marco %A Leuchter, Miriam %D 2023 %J Frontiers in Psychology (Front. Psychol.) %N 1270931 %P 1-17 %R 10.3389/fpsyg.2023.1270931 %T Measuring university students’ ability to recognize argument structures and fallacies %U https://pubs.wi-kassel.de/wp-content/uploads/2023/12/JML_954.pdf %V 14 %X Theory: Argumentation is crucial for all academic disciplines. Nevertheless, a lack of argumentation skills among students is evident. Two core aspects of argumentation are the recognition of argument structures (e.g., backing up claims with premises, according to the Toulmin model) and the recognition of fallacies. As both aspects may be related to content knowledge, students studying different subjects might exhibit different argumentation skills depending on whether the content is drawn from their own or from a foreign subject. Therefore, we developed an instrument to measure the recognition of both argument structures and fallacies among the groups of preservice teachers and business economics students in both their respective domains (pedagogy and economics), and a neutral domain (sustainability). For the recognition of fallacies, we distinguished between congruent and incongruent fallacies. In congruent fallacies, the two aspects of argument quality, i.e., deductive validity and inductive strength, provide converging evidence against high argument quality. In incongruent fallacies, these two aspects diverge. Based on dual process theories, we expected to observe differences in the recognition of congruent and incongruent fallacies. Aims: We investigated whether these two abilities are domain-specific and whether the recognition of fallacies depends on the congruence of two aspects of argument quality. Methods: 267 preservice teachers and 56 business economics students participated in the study. For the recognition of argument structures, participants assigned the five statements constituting one argument to the corresponding component according to the Toulmin model. For the recognition of fallacies, we created arguments and incorporated a common fallacy into some of them: formal fallacy, overgeneralization, irrelevance, or circularity. Participants rated whether the argument was cogent or not, which was followed by a brief justification. Results: Domain specificity could not be found for either of both abilities. For the recognition of fallacies, two dimensions were found: a congruent dimension (formal fallacies and overgeneralizations) and an incongruent dimension (irrelevance and circularity). Discussion: The instrument measures the recognition of both argument structures and fallacies in these two groups across domains. The recognition of fallacies differs depending on whether the deductive validity and the inductive strength of the argument are equally indicative of argument quality or not.