The workshop is part of the ACM Conference on Fairness, Accountability and Transparency (ACM FAT*) which will take place on January 27-30, 2020. During conference GeDIS team invites you to join one of the CRAFT sessions: "Lost in Translation: An Interactive Workshop Mapping Interdisciplinary Translations for Epistemic Justice". The CRAFT (Critiquing and Rethinking Accountability, Fairness and Transparency) sessions are open for conference participants. Conference registration is still open until January 20, 2020.

If you would like to participate in the sessions, you can already get involved - read more about it on this Medium.com post and contribute your thoughts to the Cryptpad.

Lost in Translation: An Interactive Workshop Mapping Interdisciplinary Translations for Epistemic Justice

Organisers and presenters: Phillip Lücking, Goda Klumbyte - Gender/Diversity in Informatics Systems, University of Kassel; Aviva de Groot, Evelyn Wan, Mara Paun, Shazade Jameson - Tilburg Institute for Law, Technology, and Society, Tilburg University; Danny Lämmerhirt - Locating Media Graduate School, University of Siegen

Abstract

There are gaps in understanding between those who design systems of AI/ ML and those who critique them. This gap can be defined in multiple ways - methodological, epistemological, linguistic, or cultural. To bridge this gap would require a set of translations: the generation of a collaborative space and a new set of shared sensibilities that traverse disciplinary boundaries.  We propose a workshop which aims to explore translations across multiple fields, and translations between theory and practice, as well as how interdisciplinary work could generate new operationalizable approaches. Through 3-hours of joint discussion and interactive exercises, the workshop will generate insights regarding the challenges of interdisciplinary work, identify shared concerns and pressing issues facing our field(s), and pave the way for actionable steps to introduce change in our respective practices that would be conducive to future interdisciplinary collaboration. This to us would help achieve a vision of epistemic justice, where new grounds for interdisciplinary knowledge could be built, validated, and adopted towards the goal of FAT* and beyond in computational systems.

Description

'Knowledge' is a social product which requires fair and broad epistemic cooperation in its generation, development, and dissemination. As a marker for truth and therefore a basis for action, it is inherently loaded with morals. This defines important values for our epistemic practices - in our case, as researchers. As a community that is well aware of its need to produce reliable output, how can we continue to cultivate more reflexive epistemic practices in the interdisciplinary research setting of FAT*? 

Philosopher Bernard Williams succinctly translated what he identifies as two core values, accuracy and sincerity, into important virtues: "take care, and do not lie." Others have registered this endeavor as 'epistemic justice': a virtue itself to prevent epistemic injustice. Fricker (1998, 2007) defines this as a specific dimension of injustice that exists when people are wronged either as a knower ('testimonial injustice') or an epistemic subject ('hermeneutic injustice'). Enabled by epistemic credence, authority or knowledge, epistemic power can be an important driver of, but also result from, other (economic, political) powers. When rational authority and social power come apart, we have a problem. Epistemic justice then, as also defined by Geuskens (2018), is the proper use and allocation of epistemic power; the inclusion and balancing of all epistemic sources. As Jasanoff (2017) reminds us, any authoritative way of seeing must be legitimised in discourse and practice, showing that practices can be developed to value and engage with other viewpoints and possibly reshape our ways of knowing.  

Particularly for this workshop, this means that the standards we set and the methods we carefully craft to get to knowledge should serve us all and exclude no-one. But as abundantly shown by critical theorists (feminist, post- and de-colonial, indigenous, critical race studies, amongst other) failing quality of both have resulted in exclusion from knowledge communities to the detriment of the excluded, the communal knowledge space we all rely on, and society in general.

The fact that knowledge of AI is concentrated in the hands of too few (certainly in light of its widespread pervasive applications) is an injustice much addressed within the broader FAT* community. Opening up the AI community, but also embedding its research in more diverse, multi/interdisciplinary settings is called for - loudly. How can we do our epistemic best in a multidisciplinary setting?

Successful epistemic cooperation at least entails being able to scrutinize each others' knowledge practices, by creating ways of knowing methods. We need to explain these in a way that others’ critique may help us improve our own understanding. Gaps in understanding between those who design systems of AI/ ML and those who critique them therefore poses serious issues. Bridging methodological, epistemological, linguistic, or cultural gaps require a set of translations: the generation of a collaborative space and a new set of shared sensibilities that traverse disciplinary boundaries. That new space needs to be developed wisely and continuously, as crossing these disciplinary boundaries is notoriously difficult (Law 2004). An example (Thorne et al, 2002): when qualitative research entered the health field, after necessarily explaining a lot methodology-wise, certain qualitative methodologies became aligned with the studies of certain diseases, thereby drawing attention to specific aspects of the diseases, mobilising resources for particular aspects of the disease and ‘enacting’ them in future research. For example, diabetes studies 'were drawn to' grounded theory, while MS researchers were more likely to draw on narrative methods. This in turn influenced subsequent research designs, such as what features of these diseases to focus on.

In short, interdisciplinarity is easy to talk about, but the practice is when you get down in the weeds. We need more specific experience from practice about the differences in assumptions, values, perspective, and on how these feed into our epistemic reasoning as well as disciplinary formats and practices of design. Sometimes we say the same words but mean different things. How can we hack our systematized research and design patterns towards new, communal methodologies?

The presenters in the workshop cover a range of disciplines including social sciences, feminist and postcolonial theory, computer science, new media studies, law and multi-scalar governance. We propose a workshop which aims to explore translations across multiple fields, iterative translations between theory and practice, as well as how interdisciplinary work could generate new operationalizable approaches. For instance, how could critical theory or higher level critiques be translated into and anchored in ML/AI design practices - and vice versa? What kind of cartographies and methodologies are needed in order to identify issues that can act as the basis of collaborative research and design? How can we (un)learn our established ways of thinking for such collaborative work to take place?

Through structured joint discussion and interactive exercises over 3 hours, the workshop will generate insights regarding the challenges of interdisciplinary work, identify shared concerns and pressing issues facing our field(s), and pave the way for actionable steps to introduce change in our respective practices that would be conducive to future interdisciplinary collaboration. This to us would help achieve a vision of epistemic justice, where new grounds for interdisciplinary knowledge could be built, validated, and adopted towards the goal of FAT* and beyond in computational systems.

Structure of workshop

The workshop will be structured in three parts followed by a plenary wrap-up and reflection session. First, there will be a brief introduction to the concept of epistemic justice and how it pertains to the interdisciplinary work that FAT* does. Rather than trying to define an arbitrary baseline to evaluate epistemic justice, the presenters will share their own experience with translating between theory and practice, disciplinary backgrounds, languages and cultures as a reflection on epistemic differences. Coming from different disciplines, the presenters will also talk about their respective workflows, i.e. practices of knowledge making. 

These will present examples of workflows from different disciplines, such as socio-cultural critique (with highlight on feminist and de-/post-colonial critique) computer science and law, which will serve as supplementary input into the next exercise. These preliminary presentations will act as a broad brush introduction to some of the different perspectives with their different areas of value and emphasis, and provide practical examples of the challenges, assumptions, possibilities and limitations of working in translation. There will be a short round of questions and discussion, concluding the first part.

We will then lead two interactive exercises in small groups. We will strive to make sure that each group has a diverse disciplinary composition, so as to facilitate the presence of different voices.

The first exercise will be “Charting methodological workflows”. The session will be premised on insights from the history of science (e.g. Feyerabend 1975), anthropological inquiries into ‘multiples’ (Mol 2003), and the notion of the social life of methods (Savage 2013). We will argue that methods never merely represent, but always perform social realities (Law 2004) and shape how we see worlds (Uprichard and Dawney 2016), and that phenomena like ‘the algorithm’ are at the same time brought into being in very different ways by different actors depending on their resources of meaning-making (Seaver 2017). A critique of epistemic practices must therefore not only attend to how methodology constructs social worlds, but also to how people are equipped with different resources to see, enact, stabilise, and critique it. 

Based on this premise, as a starting point for discussion in the small groups the exercise will begin by presenting a prototypical workflow of designing algorithmic systems, including choice of databases, use of statistical methods and so forth. Drawing from participatory design and critical data methods such as data sprints, we will elicit different ways of thinking and collaborating in knowledge production and disciplines. We will encourage people to make their own workflows visible. We wish not to only think ‘inside’, or ‘with’ the algorithmic design workflow, but also in dialogue with workflows contributed by the participants, resulting in a variety of approaches. Whether this is iterative, inside or outside of algorithmic design workflows can be left open for the small groups to explore what they find a priority. Multiple spatial and aesthetic configurations of expression will be encouraged. The exercise will be geared at identifying intersections, overlaps or gaps, which open up moments in which translations and interdisciplinary interventions between these different workflows or approaches could happen. The goal is still to think about technical workflows, but diffracting them through critical approaches that are rooted in epistemic justice. Such a mapping of potentially interrelated co-creations could then be used by people from a variety of disciplines. Conclusively, the outcome of this exercise will be an array of de/reconstructed workflow models, as well as descriptions of potential mappings of interdisciplinary translations. This opens up the potential where other ways of seeing algorithms can come into dialogue. The exercise is intended to be a practical example of a generative task as an experience of translation, while also generating a shared terrain of intervention. Participants are encouraged to note down their reflections as they go, which will also contribute to the collaborative documentation.

The second exercise will be “Translation cartographies”, that will serve as a structured reflection on the practices of translation that happened during the operationalisation of the model. Groups will be invited to reflect on their own interdisciplinary process of having created their critical workflow model. It will start by laying out what are the backgrounds of the people at the table on the bottom, and the ‘outcome model’ at the top. Each group will then draw trees, lines, or shapes upwards, to represent the journey of the translation process as a group. The branches and nodes represent what was assimilated, joined, or got lost in translation. What happened in their conversation? What translations were made, what was left out? Were there critical moments which surfaced, such as understanding massive or subtle differences in meaning or similar words? Which tensions emerged? Participants will be invited to end the exercise with reflecting back on their own practices. The workshop will end with a group discussion and “what to do next” session.

Throughout the workshop two strategies will be used to facilitate the space for different voices: first, we will use a set of anti-oppressive facilitation techniques that have been developed across social justice activist circles. Second, on a conceptual level, we will also highlight perspectives that are rooted in anti-racist, feminist, post- and de-colonial knowledge practices, as well as critical approaches in law. This will be done already through initial input presentations by the organisers, as well as during presentation of different workflows.

Outcomes

Each phase of the workshop is structured to produce tangible artefacts, which serve as outcomes as well as jumping off points for the session. As an evolving mapping, these are how people’s engagement is made visible. The less tangible outcomes are a more tightly articulated understanding of the differences across vocabularies and languages and the challenges of translation, and the experience of what that work of translation could look like in practice though the workflow exercise. The concluding summaries will also serve as suggestions for the broader community. A very concrete outcome would also be identification of the actionable points for each of the participants’ practice.

Glossary of main concepts