From needs assessment to usability testing: evaluating the AinoAid™ chatbot for domestic violence support

Violence against women and girls (VAWG) is defined by the Istanbul Convention as “a manifestation of historically unequal power relations between women and men, which have led to domination over, and discrimination against, women by men and to the prevention of the full advancement of women” [1]. In particular, domestic violence (DV) by current or former intimate partners in any form has the potential to be deeply traumatic for those females who experience these acts. The consequences of DV range from physical injuries [2] and insomnia [3] to depression [4] and even suicide [5]. Approximately 47,000 cases of domestic homicides were counted worldwide in 2020 [6]. In the European Union (EU), 17.7% of surveyed females have experienced physical violence or threats and/or sexual violence by an intimate partner in their lifetime [7]. As a consequence, the estimated economic costs of DV are calculated by the Council of Europe [8] to be 555 € per EU citizen annually.

Nonetheless, the epidemic of DV remains largely hidden, as perpetrators actively conceal their actions, for example, by manipulating survivors of DV into believing that they are solely responsible for and deserving of the violence they endure [9]. Thus, a crucial gateway to support lies in enabling survivors to identify violence and act despite doubting their perceptions, their fears and the enormous taboo that surrounds violence in a formally loving relationship. Accordingly, research shows that survivors’ ways to find protection and support are blocked by individual, situational and structural barriers [10]. Individual barriers pertain to survivors’ perceptions, expectations and experiences and influence whether survivors are able to seek help. Self-blame, low self-esteem, shame, the perception of violence as a taboo and a private matter or as a “man’s right” lead survivors to belittle violent incidents and their consequences [11]. Survivors may lack knowledge about support services, which leaves them in a passive position [12]. Moreover, the survivors’ ability to seek support is impaired as they are scared, stressed, and traumatized, which decreases their ability to function [13]. Situational barriers pertain to circumstances, current conditions, social relationships, etc., that influence whether survivors are able to obtain support. Perpetrator behaviours such as ‘gaslighting’, social isolation, and intimidation keep violence concealed [14]. Similarly, survivors’ spatial and social isolation [15] and their responsibility for their children [16] bind them to stay in abusive relationships. Furthermore, strong social norms, such as acceptance of violence, partnership idealization or fear of stigmatization [17], hinder survivors from breaking the taboo of their situation. Naturally, the consequences that survivors expect to face when disclosing violence to a trusted person, make the violence public or splitting up are extremely fear-laden [18]. This includes the fear of not being believed [19], of being ignored, of not being taken seriously, of unpleasant questions, in addition to the fear of consequences such as losing social capital, becoming socially isolated [20], or suffering other social repercussions [21]. Structural barriers are system-related obstacles that affect the provision and quality of protection or support services day-by-day or in crisis situations [22]. Some survivors carry negative perceptions and experiences of society, support systems and institutions that lead to a low level of trust [23]. Those with a migration history may fear that they lose their residency permit when seeking help [24]. In addition, the ‘Group of Experts on Action against Violence against Women and Domestic Violence’ (GREVIO) [25] regularly criticizes the inadequate availability and distribution of services. Thus, survivors need to be informed about what to expect from which service to assert their rights when faced with a lack of understanding or proactivity [26].

In summary, women affected by DV find themselves in a vulnerable position, which might even be exacerbated by marginalization and intersectionality. Thus, DV survivors need 24/7-available support and valid information as well as empowerment and encouragement to dare to seek and receive support. In many locations, networks of support services exist [27], but survivors need more information and support to find the gateway and overcome the threshold between being victimized and taking concrete action. Owing to their vulnerability, it is important that their autonomy in decision-making is maintained such that they can choose their pace and their direction of help seeking, where they do not have to fear unpleasant questions. This aspect is particularly relevant for individuals who have experienced severe trauma.

The aim of our research and innovation project is to harness artificial intelligence (AI) to provide survivors, as a particularly vulnerable group, with a low-threshold gateway to protection and support. AI has the potential to overcome, mitigate or circumvent the manifold barriers DV survivors face. Nonetheless, the current literature has critically examined, whether AI can provide a positive solution, and in what ways. Studies indicate that DV survivors express concerns about the indispensability of human support, in addition to issues of safety and security concerns. Empathic connections and trust-building are more difficult to establish due to the absence of social cues, and technological challenges may also arise. These include issues of connectivity and privacy, simultaneously creating the need for access to technology and the internet [28, 29]. While some survivors do not have access to technology, other minoritized or marginalized survivors (e.g., neurodiverse, older, deaf, or struggling with literacy) may face barriers to accessing technology without further help [30]. In parallel, it also highlights its potential for anonymity, a non-judgemental approach, immediate, accessible responses, overcoming language barriers, the provision of additional avenues to safety, and the opportunity to establish a central recording system for criminal processes, and concludes that it may be “better than nothing” [29‐31]. Although some persons might be reluctant to use text-based services, most individuals do not differentiate in their disclosure preferences between an AI and a human counterpart [32]. After their introduction, some text-based crisis hotlines are used as often as telephone hotlines [31, 33]. When provided in a timely manner and in ways that properly identify users’ needs and options, technology can be a valuable support and virtual front door to services [29, 34]. By offering information on resources and support, and by contributing to survivors’ understanding of their situation and to their improved safety, it strengthens access to help. This pertains especially to young persons and those living in remote areas, as well as to situations where direct human contact might not be feasible or accessible (e.g., due to transportation problems, child-care responsibilities, lack of time or pandemic conditions). In this way, more survivor groups can access the support system, for instance, individuals who want to avoid the stigma of seeking help or who do not wish to be seen doing so [28, 29, 34‐36].

Accordingly, digital services need to be effective in rapport building and therefore must mimic unconditional positive regard, active listening, asking questions, paraphrasing, validation, and evaluative language specifically tailored to the user input [28]. Moreover, digital service in the field of DV need to accurately recognize users’ expressed sentiments or emotions – particularly in those with depressive symptoms – and react appropriately, while also conducting risk detection [37]. In parallel, service providers also need to support these tools to encourage their use in practice [38],, e.g., by ensuring that clients use high-quality, trusted, and secure digital tools if they discontinue using the providers’ services. In this regard, the psychological risks of AI, arising from misleading pseudo-intimacy, have also been widely discussed, as they pose a particular threat to users who experience isolation or psychological distress [39]. The listed risks are manifold, ranging from commodified intimacy, emotional over-attachment, emotional dependence, emotional delegation, the replacement of relational labor by algorithmic simulation, manipulative engagement, and reinforcement of social isolation, to risks related to data extraction, consent and boundary violations, ethical roleplay violations, deceptive outputs, and potential misuse. Further risks include biased training data or misinterpretations, lack of maintenance and development of the technology (e.g., outdated content or broken links), and ethical risks arising from vulnerable users or unregulated design, which can lead to unintended consequences [30, 37, 39‐41].

To manage these risks, digital services must be critically and regularly evaluated for their potential to undermine users’ emotional capacity for action and for their safeguards that preserve transparency and accountability, ensuring that users are not exposed to further risks (e.g., perpetrators’ control of survivors’ devices for abusive reasons) [30, 39]. Nonetheless, research shows that users of human-like chatbots indicate that, despite these risks and potential harms, these digital “relationships” are beneficial to their well-being. The more consciously human-like the chatbot is perceived, the more positive opinions and pronounced social health benefits are reported [42].

Responding to recent research appeals [43, 44] and a call by the EU [45] to develop an AI chatbot for DV survivors as an intervention to bridge the gaps between victimization and active help-seeking, we developed the AinoAid™ chatbot [46] as part of the larger EU project ‘Improving Access to Services for Victims of Domestic Violence by Accelerating Change in Frontline Responder Organisations’ (IMPROVE) [47], which we introduce in the following section. Our primary research objectives are as follows:

AinoAid™ [46] is an AI-based service designed to provide psycho-social support and guidance for survivors of domestic violence (DV), acting as a gateway to protection and support, as well as assisting frontline professionals in offering effective support and guidance. Moreover, AinoAid™ is going to be enlarged to further VAWG content within the next 2 years in the framework of the REACH project. This in particular pertains to content on sexual violence, forced marriage, and human trafficking.

The core premise behind the design of AinoAid™ was to create an accessible, reliable, and trustworthy resource where users can find, access, and utilize information on DV [44]. AinoAid™ [46] is compatible with smartphones, tablets, and computers and does not require any downloads. The service consists of two main components: (1) an AI-based chat designed to provide support and guidance and (2) a knowledge base that offers valuable information. AI-based chats aim to provide psycho-social support by mimicking human conversation, and were co-created and curated with input from DV survivors, therapists, and psychologists to reflect interactions typically found in therapeutic settings [43], with strict adherence to safety and ethical principles. The content of the knowledge base, created by DV experts from the IMPROVE consortium, can be accessed either through the chat interface or independently without engaging in a conversation. It includes frequently asked questions (FAQs), safety guidance, detailed information on various forms of violence and on how to recognize these, instructions on seeking help, information on legal and criminal processes, on accessing shelters and support services. The platform also offers checklists, a safety plan template, a self-risk assessment questionnaire and contact information for emergency services and other relevant organizations, such as mental health support.

AinoAid™ has been launched in Austria, Finland, La Réunion, Germany, and Spain. Accordingly, the content has been tailored to the specific national contexts and DV-related legislation. When accessing AinoAid™, users are first prompted to select their country and then the language they wish the platform to refer to [46]. When a user queries the chatbot, it accesses the vector database linked to the Sanity CMS [48], fetching the most relevant content (i.e., operating on a Retrieval Augmented Generation/RAG approach). The system prompt then adds instructions regarding tone and style (e.g., empathetic, supportive), and the language model combines the user’s message, the chat history, and the retrieved content to generate a response. If the risk-classification or intent-detection models flag the need, a contact card (with helplines and local services) is added automatically. The prompting instructions ensure that the model’s tone remains empathetic, professional, and supportive. In addition, established processes for continuous monitoring and feedback collection during piloting and beyond enable concerns about tone or user-friendliness to be identified early and addressed promptly. This ongoing monitoring helps the chatbot to respond to users’ needs in a manner that is both effective and sensitive.

To guide the dynamic and iterative development process of AinoAid™ [46], an interview study with DV survivors (Study 1) was conducted in parallel with the initial development phases of the chatbot. The goal was to assess the needs, expectations, and concerns of potential users. The findings from this study were then used to refine AinoAid™’s features, ensuring that they better aligned with the identified needs and expectations of survivors. In a subsequent follow-up study (Study 2), public opinion on the chatbot was surveyed to evaluate how users rated the key features of AinoAid™ that were highlighted as important by participants in Study 1.

New digital technologies, such as the AinoAid™ chatbot, have immense potential to improve the support and protection of survivors of domestic violence [36]. Technology can expand the reach of interventions by increasing awareness and providing knowledge, thus encouraging survivors to seek support from appropriate authorities and services [29‐31].

We evaluated the potential and usability of the AinoAid™ chatbot through two studies. In the first study, interviewees from different countries shared their views on the chatbot’s potential and desired features, while in the second study, a group of potential users and professionals assessed the usability of AinoAid™ and provided ideas for its further development. Most interviewees viewed the chatbot positively as a potentially significant source of information and support for survivors of DV, but also raised concerns. The second study involved a group of 669 females who tested AinoAid™ and evaluated its usability and language quality. Overall, the respondents rated the usability of AinoAid™ positively, while also asking for simplified language, detailed guidance and enabling of the chatbot to retain information shared earlier in the conversation. They also suggested incorporating the ability to contact various authorities, support services, and emergency responders directly through the chatbot, which, however, would undermine user anonymity.

AinoAid™ offers a rich content, including safety guidance, information on various forms of violence, advice on seeking support, and details about legal procedures, shelters, and other social services. By offering these resources, it serves as a crucial first point of contact, helping survivors of DV become better informed about their situation. Providing information about available services and what to expect from them is a vital first step for survivors of DV to assert their rights [26]. Moreover, AinoAid™ is designed to address some of the barriers that may prevent survivors of DV from seeking help from various service providers. These barriers often intertwine, creating unique constellations in each individual case. For example, individual psychological factors are often intertwined with so-called situational barriers. Among these situational barriers are the cultural context and social relationships in which a survivor’s life is embedded. A survivor of violence, for instance, may experience social isolation when persons in her social network are inclined to accept norms that tolerate violence and the subordination of women [17]. Survivors living in such cultural and social contexts may find AinoAid™ reinforcing their perceptions of their rights and the reprehensibility of violence. However, reporting violence to authorities or relatives, for instance, may worsen survivors’ social isolation if friends and relatives disapprove their actions [46]. In such situations, AinoAid™ can give, nonetheless, advice on self-care, networking and encourage the survivor.

AinoAid™ can help reduce survivors possible lack of awareness of their situation and the lack of knowledge about the availability of various services. Among the most severe barriers are factors that cause survivors to doubt their experiences, leading to self-blame, shame, and low self-esteem [12, 13]. In such circumstances, a conversation with AinoAid™ can help alleviate stress and fear, while improving the survivor’s ability to seek support from various services.

There are structural factors related to the system’s capacity to provide various types of support services and measures that contribute to the survivor’s welfare, health, and security, e.g., geographical distance to services can act as a tangible barrier. Similarly, the lack of services for specific needs, such as support for women with children or the unavailability of interpreters, may limit the survivor’s access to certain services. Negative experiences with particular service providers may, moreover, lead to distrust and reluctance to seek further support [23]. In fact, AinoAid™ assists survivors in establishing realistic expectations regarding the social, medical, and judicial services available to them. However, AinoAid™ cannot address situations where services are simply unavailable in certain areas or for specific groups. There are, nonetheless, also survivors’ misconceptions about the quality and availability of services. These misconceptions can be corrected through the information provided by the AinoAid™ chatbot. Furthermore, even if a survivor has had negative experiences with a particular service provider, AinoAid™ can encourage the survivor to re-contact authorities, regardless of previous negative experiences.

Accordingly, the key factor in the success of the AinoAid™ chatbot is whether the service system can accommodate the potential increase in demand for services. While AinoAid™ can stimulate greater demand for DV-related services, including social services, healthcare, law enforcement, and third-sector support, if these services are unavailable or if their quality does not meet users’ expectations, AinoAid™ may ultimately be perceived as disappointing.

Naturally, the results of this research are subject to further limitations. As the AinoAid™ service is primarily targeted at survivors of domestic violence, they are only presented partly within the sample of Study 2. However, the development of AinoAid™ is an iterative process, meaning that an initial version of AinoAid™ would still be in need of user feedback to improve it towards being appropriate for survivors. To not burden this vulnerable group with a potentially premature service, this group should for ethical reasons be included only at the end of a process for fine-tuning the service, when the chatbot is best trained and all functions are working properly. Instead, general aspects such as usability and most aspects of chatbot language can be equally evaluated by persons not affected by domestic violence. AinoAid™ is, moreover, also conceptualized to support survivors’ friends or bystanders and professionals working in the field of domestic violence. As AinoAid™ has received further training since the survey took place and feedback was used to inform further service development, future research should investigate how survivors who use AinoAid™ in times of need evaluate the service and whether the service has the power to motivate them to approach services of the help system, as intended. In addition, future research should examine the perspectives of DV frontline professionals (e.g., advocates, case managers, and counsellors) on AinoAid™. Such insights would help evaluate the suitability and accuracy of the tool’s content and presentation for DV survivors and clarify whether the service sets realistic expectations of the support system.

The use of technological tools, such as an artificial intelligence chatbot, to improve survivors’ access to support services cannot be implemented without addressing the ethical and security concerns that survivors may have when utilizing such AI-based services. An unconditional prerequisite for the use of the chatbot is its anonymity [32]. Technology must ensure that survivors’ privacy, data security, and safety are not compromised [39, 47]. A second important factor is the immediate and constant availability of the chatbot. This feature is crucial for survivors who are controlled by the perpetrator and may only be able to contact the chatbot occasionally [34]. Besides, a key strength of AinoAid™ is its non-judgmental, impartial nature as the content of the chatbot is curated to not being manipulative, moralizing or accusatory [40]. AinoAid™ is built upon both professional expertise and the experiences and insights of survivors. It provides information and support that are both accurate and relevant for various types of DV survivors. Each survivor is unique and follows her own pace and direction when seeking help. Accordingly, AinoAid™ offers diverse and comprehensive content to meet the needs of a wide range of users, helping them access information and support [39]. A final advantage of an AI-based chatbot is its ability to raise awareness not only among survivors but also to challenge normative and cultural discourses that may attempt to legitimize DV, oppress women, or even blame them.

This research contributes to the growing body of research exploring the use of technology as a tool for the empowerment and support of women who are survivors of domestic violence. Developed within the framework of the EU project IMPROVE, the AinoAid™ AI-based chatbot demonstrates potential as a valuable source of information and assistance for DV survivors. Our evaluation confirms that AinoAid™ functions as a safe, accessible, and effective gateway to protection and support for survivors of DV. However, ensuring user safety and anonymity, particularly in the context of digital technologies, remains critical for enabling individuals to seek advice and support through AI-based chatbots. The key finding of Study 1, which showed survivors’ openness to the concept of a support chatbot and their positive view on its potential, highlights the value of such tools in complementing existing support services for DV survivors. This is further reflected in the positive evaluations of AinoAid™ in Study 2.

One of AinoAid™’s greatest strengths is its impartiality, which stems from its non-human nature. While this impartiality is highly valued, it also presents a significant challenge in meeting the high expectations for human-like, empathetic, and dynamic conversations in such a sensitive context. Nevertheless, we anticipate that sustained usage and continuous training will enhance the AI’s ability to fully meet the evolving needs of DV survivors.

Despite the valuable contributions of these studies, further research is needed to fully understand the impact of AinoAid™ in addressing and preventing DV. Specifically, examining the perspectives of professionals who interact with the chatbot could offer critical insights into its effectiveness and practical implementation. By incorporating feedback from all potential user groups, AinoAid™ can be further refined to serve as a more comprehensive and effective tool for both, immediate crisis support and long-term intervention.