Invisible risks: Combating secondary trauma to safeguard children

More than 100,000 people worldwide are content moderators. The job involves analysing material that has been flagged as offensive or harmful and deciding whether it should be removed from the platform it was uploaded to or not. The very nature of the job means content moderators are at risk of being exposed to traumagenic content, including child sexual abuse material (CSAM).

Content moderation can be a demanding role, potentially leading to high levels of burn out, secondary trauma and mental health difficulties. These negative outcomes can decrease work standards, thus failing the people they seek to protect. Despite this, primary research on the psychological effects of content moderation in terms of wellbeing, productivity and vulnerability to negative outcomes is lacking.

Secondary traumatic stress (STS) is characterised by a set of reactions, such as feelings of isolation, dissociation and sleep disturbance, in response to working with traumatic material. As such, content moderators who deal with large amounts of accounts or images of abuse would be expected to be at increased risk of developing STS. Indeed, repeated, prolonged exposure to specific content, coupled with limited workplace support and counselling, can significantly impair the psychological well-being of moderators. Additionally, there is work suggesting that secondary traumatic stress is often comorbid with anxiety and depression. Therefore, it is likely repeated exposure will increase moderators’ risk of developing anxiety, depression and stress disorders. If left untreated, this can lead to absenteeism, lower quality of life, burnout, and work dissatisfaction. Alongside continued exposure to traumagenic content, moderators can face other pressures such as needing to hit high accuracy and throughput targets. This can exacerbate psychological discomfort and can cause burnout.  However, despite this there is a lack of primary research into the psychological impacts on content moderators and what can be put in place to support them.

The project uses an iterative approach to develop a model of content moderator needs and an intervention to support and build upon areas of resilience. There are four main stages:

STAGE 1​

Theory of change workshop will be conducted to develop a detailed set of intermediate and final outcomes that stakeholders share as relevant in the support and wellbeing of content moderators​

STAGE 2​

In depth interviews will be carried out with individuals who have previously or currently work with CSAM as part of their job in content moderation or parallel professions​. Interviews will aim to capture the complex experiences of working with CSAM, its impact, how these are mitigated individually and by the organisation, as well as positive and negative elements of the role. ​

STAGE 3​

Anonymous surveys will be distributed to current content moderators on the potential adverse effects of their role, coping strategies employed and organisational sources of support and stress ​

STAGE 4​

Based on the findings from the previous stages an online support package will be developed that is specifically tailored to address areas of stress and build on areas of resilience that are specific to content moderators and their role

A small sub-sample of content moderators will be invited to pilot the support package and complete surveys pre- and post- support package so we can assess its efficacy

Participants will be given the ability to opt in to a short interview about their experience of using the intervention​ so we can get feedback about the user experience, suitability and potential improvements.