The research project “Secure self-disclosure in intimate communication with dialog systems” (SENTIMENT) deals with intimate communication with natural language dialog systems. These are coming ever closer to authentic and human-like communication. As a result, the boundaries between a definable artificial intelligence and an anthropomorphized communication partner are becoming blurred. There are also more and more applications that deliberately use the new capabilities of language models to actively build relationships with users and thus obtain particularly sensitive information. The naturalness of the language of such systems is the key to transferring social norms and, accordingly, communication patterns to artificial communication partners.

One mechanism that is particularly worth investigating here is self-disclosure: people tend to reveal more information about themselves if the other person also shares something personal. Dialog systems also systematically use the information obtained to make communication appear more individual. Personalized language models are able to create user profiles based on past interactions and incorporate words frequently used by users into the communication. Conversely, such personalized communication can strengthen the bond and trust to the counterpart and contribute to even more intimate communication. The artificiality of the counterpart also offers a supposed sense of security from potential social consequences such as a lack of understanding, shame, a need to explain or restrictions on contact, especially when it comes to particularly intimate and sensitive topics that are less openly discussed or accepted in society. Accordingly, these interactions also generate a huge amount of extremely sensitive data.

Studies with users of such dialog system-based companion apps show that data protection concerns on the part of the users of such applications are pushed aside by emotionally influenced considerations, i.e. the processes of self-disclosure, trust and relationship building and positive confirmation of the users described above. This puts users in vulnerable situations. Ultimately, extremely sensitive data is collected on the basis of the basic human need for social belonging and can thus be passed on to third parties (more or less anonymized). Thus, the project deals with the question of how to ensure adequate protection for users when dealing with intimate communication technologies.

The legal sub-project deals with questions of the legal requirements for the protection of privacy and self-determination at constitutional and statutory level, with the regulation of artificial intelligence and develops practical recommendations for the design of technology and for the further development of law in specific areas.

A special feature in this research project is the collaboration with the Kunsthochschule Kassel to reflect on the methods and research approaches as well as the content-related questions, topics and results of all partners from a design and art perspective and to make them tangible.

The SENTIMENT project is the subject of the funding guideline “Platform Privacy - IT Security Protects Privacy and Supports Democracy” as part of the German government's research framework program on IT security “Digital. Sicher. Souverän” (“Digital. Secure. Sovereign”). The BMBF is funding the project for three years with € 1.24 million.

Partners:

• Universität Duisburg-Essen, INTITEC (Intimacy with and through technology) Junior Research Group, Dr. Jessica Szczuka (Koordination),
• Kunsthochschule Kassel, Professorship for New Media, Prof. Dr. Joel Baumann,
• Ruhr-Universität Bochum, Faculty of Computer Science, HGI and Excellence Cluster CASA, Dr. Veelasha Monsaamy and Dr. Theodor Schnitzler (Assistant Professor at Maastricht University, Department of Advanced Computing Sciences).

For further information see the BMBF website.

Funding:
Federal Ministry of Education and Research

Duration:
April 2024 - March 2027

Project leader:
Priv.-Doz. Dr. Christian Geminn

Staff:
Dr. Maxi Nebel