An online network tool for quality information to answer questions about occupational safety and health: usability and applicability

Many people have questions on health or health-related issues, such as occupational health and safety (OSH) matters[1‐3]. Although people in the Dutch working population have the legal and moral right to high quality answers, probably more than one million OSH questions remain un- or incompletely answered annually[3]. In theory, a knowledge infrastructure should provide high quality information (i.e. usable and evidence-based) to answer these questions through several facilities: information knowledge products, such as fact sheets or guidelines for practice; expert advice, such as from occupational physicians; and education and training by the company or as part of vocational training[4]. Clearly, the OSH infrastructure needs new or improved facilities or strategies that account for the barriers questioners in the working population experience in finding and using high quality information.

Specific research on the effectiveness of particular facilities or strategies for answering OSH questions of workers is absent. Almost all research on strategies to answer OSH questions is oriented towards professionals and the use of evidence-based practice strategies (EBP)[5, 6]. Although the EBP strategy has been proved effective for OSH professionals, this strategy is not logical for the working population, as this strategy is time consuming and workers are not familiar with the terminology used in professional and scientific publications. Therefore, other strategies or facilities would probably be more suitable[1, 7‐9]. First, information and knowledge products are often numerous, free of charge and easily accessible, but these are not always specific or updated, and quality is regularly lacking. Second, OSH experts can provide high quality, and tailored answers quickly but are often not easily accessible or free of charge. Finally, education and training facilities could provide or support the finding of high quality answers, but they are time consuming and sometimes expensive. An interesting new strategy could be to combine useful elements of existing facilities through providing an online link between the person who is searching for information and an expert who provides tailored, high quality answers, possibly without charge.

Today, web-tools, such as patient forums, social networks (e.g., Facebook, LinkedIn) and Question and Answer (Q&A) network technologies, can establish such links. With these tools, an easily accessible network of experts answering questions can be created. Online Q&A network tools seem particularly promising for communication, information exchange, information storage and information retrieval[10, 11]. The technology has already been applied in some large knowledge-intensive organisations, such as Philips and ABN AMRO Bank[10]. For these organisations, it is essential that experts easily find each other to exchange specialised information and knowledge. With a Q&A network tool, users may find a specific expert for their specific health or health-related problem and ask their question directly to that expert. The tool facilitates this process by sending an e-mail notification to the expert (when questioned) and the questioner (when answered). Questions and answers are stored in a searchable database for public re-use. To avoid privacy issues, a moderator can remove privacy-sensible information from the question or answer or prevent public access to the database before publication.

Although several models have described the process of designing and testing tools to suit the purpose of an intended new setting or context, many of these models included user-developer interactions[12]. When developing interactive information tools, the user-centred design of the International Organisation of Standardisation (ISO) is often applied (ISO 13407)[13]. A fundamental concept in this design process is usability[13]. Usability is defined as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use (ISO 9241-100; 2009)"[14]. Usability is associated with high website satisfaction, use[15] and loyalty[16]. In addition to usability, we believe the perceived applicability of a new information tool is important for future use. In this study, we defined applicability as the perceived capability of a new tool to provide quality information to specific target groups under specific user conditions. Therefore, assessing both usability and applicability during the development of an online Q&A network tool is important.

In this study, the usability and applicability of the prototype of the online Q&A network tool ArboAntwoord http://​www.​arboantwoord.​com was tested. This study is the first in a series of studies on the added value of the ArboAntwoord website within a given OSH infrastructure. The tool was created for all workers in every industry and sector who encounter difficulties in finding quality answers for their OSH questions. The website was launched through small-scale campaigns in which several articles were presented in national OSH magazines and websites. At launch in October 2008, 71 national experts in the field of OSH committed to the project and started answering all types of OSH questions from workers. The question topics were diverse, ranging from possible health risks of working with specific chemicals to return-to-work interventions for women with breast cancer and from work-climate law and regulations to safety solutions for working alone in small closed spaces. The aim of this study was to investigate the usability and applicability of an online Q&A network tool related to OSH for the intended user groups.

The findings of this study showed that most features of our prototype Q&A network tool were usable, although some of them could be improved. The majority of the tasks were executed effectively, whereas task efficiency and satisfaction varied. Participants helped to identify various possibilities for improvement, including features such as the process of asking a question, searching for an answer by category, obtaining technical help from the moderator or help function and expert registration. As a result, in the revised version of ArboAntwoord, launched in October 2008, we limited the amount of scrolling and added tracking (steps) to the questioning process, allowed questioners to select experts themselves, redefined (sub)categories, moved the hyperlink to the help function in the header, registered experts ourselves and presented the rules for expert participation.

The results of our study further suggested that an online network tool is an applicable information tool for the OSH field. Some questioners preferred to consult a familiar expert in as well. The tool was stated to be applicable for non-urgent health problems and for gathering additional information. The experts stated that the system might assist in observing new OSH trends and might facilitate contact between questioners from the working population and experts. Hosting and support by a trustworthy professional organisation, anonymity, timely answers and effective promotion campaigns were mentioned as important requirements for use. Usability findings and participant remarks on online Q&A target groups indicate that online Q&A network tools are not applicable for people with no or only limited computer or internet experience. To provide OSH information to this sub-set of workers, asking a question directly or indirectly through a coordinator by telephone could be an alternative.

Little is known about the applicability and usability of Q&A tools and similar online networks for high quality information and knowledge in healthcare, although similar tools, such as tele-consulting systems and patient forums, have been discussed in the literature. Q&A tools are different in some respects (i.e. they include more or less extensive network features, self-selection of experts by questioners, e-mail notification and an easily accessible public database). Notwithstanding differences, comparison is useful. Marco et al.[24] studied an "ask-the-expert-service" of a consumer-oriented website on HIV-AIDS. Despite the fact that there was only one expert answering questions, the authors concluded that there was a great demand for online "ask-the-expert" services. This opinion shared by Umefjord et al.[25] who studied a similar service for enquiries related to health or diseases. These researchers found that an ask-the-expert-service was mostly used because of anonymity and convenience. Asking the questions and viewing the answers at a self-chosen time was highly appreciated. Other important reasons for use were to become better informed, to obtain a second opinion and to present embarrassing concerns and worries anonymously. Similar reasons for use were found by Himmel et al.[26], who studied an expert forum on infertility. The importance of a "second opinion" was also brought forward as a reason for seeking tele-advice by Eysenbach et al[27], who studied patients asking questions mainly in the field of dermatology (unsolicited e-mails sent to physicians). Marco et al.[24] stated that the facilitating conditions for the success of an ask-the-expert-service were anonymity, free access and timely answers. Massone et al.[28], who studied a non-commercial tele-consulting system in the field of dermatology, concluded that these systems are promising when they are non-commercial, discretionary, multilingual and open-access in nature. Important reasons for using an ask-the-expert-service are the easy access and the additional information or second opinion about specific health issues or interventions[24‐27]. Both reasons were confirmed by several participants in our study. Other facilitating reasons for use, such as anonymity and timely answers, are also in accordance with the results in this evaluation.

A number of possible negative aspects of expert answers or online Q&A tools have been addressed in the literature as well[7, 25, 26]. Schaafsma et al.[29] showed that, with respect to occupational health issues, experts do not always provide valid answers when compared with evidence from the literature. That study also found that answers from the consulted experts that included references or sources in general were more valid than answers without such sources. Therefore, in addition to selecting experts on their knowledge, experts should be encouraged to add sources or references to their answers. Eysenbach et al.[7] warned that people could overuse ask-the-expert-services in their desperate search for additional information. Receiving too many questions can create a problem for the participating experts. The experts in our study also indicated their concern about receiving too many questions. To encourage expert participation, we provide experts with a 10 Euro incentive for each answer. In addition, we developed new features for ArboAntwoord through which experts can now do the following: 1) define the amount of questions they want to receive each month; 2) return non-relevant questions to the questioners; and 3) pass on questions to other, more suitable experts in the network. In this way, experts can regulate the amount of questions they receive. Furthermore, answers may be too complex for a questioner to understand [25, 26]. Himmel et al.[26] warned of the possibility that answers may be superficial. Either way, in ArboAntwoord, we try to prevent this by adding a feature that allows questioners to ask an "additional question" in reaction to the answer of an expert. A questioner can ask for a clearer or more thorough explanation of the first answer. Another possible adverse aspect was mentioned by Eysenbach et al.[7]: many users are sending excessively personal details over the Internet. Preserving privacy is of paramount importance for these types of online Q&A tools. Therefore, we applied a Secured Sockets Layer (SSL) for ArboAntwoord. SSL is an encrypted protocol that secures communication through the Internet. All questioners ask their questions anonymously, so no personal information is published on the web. Finally, we created a feature by which questioners and the moderator have to authorise Q&A combinations to be published in the public database. The moderator screens all Q&A combinations on suitability before publication. The moderator can remove sensitive personal information in a Q&A combination before publication, omitting or changing localities, gender, age, occupational and/or medical details. The questioner and moderator can also choose not to publish the Q&A combination at all. We think that a hosting organisation should draw special attention to legal matters and privacy policy. A disclaimer is clearly a good start but is not sufficient.

The strength of our study lies in the user-centred design used to evaluate and improve important features of this new Q&A network tool for OSH before implementation. However, the study has several limitations as well. First, the sample was limited and unevenly distributed with respect to age and sex, which may lead to overestimation or underestimation in the study results[30]. In addition, the sample was not entirely representative for purposes of assessing applicability. It would have been better to recruit a larger sample from different settings, particular organisations or occupations, who actually had (answered) OSH questions. Second, the test took place in a field setting: the participants' workplace. Possible differences in this setting such as screen size, internet connection speed and keyboard features can result in dissimilar experiences and different research results. However, the advantage of a field test is that it represents the real life situation better than a laboratory experiment. A third limitation is the data collection method. Observation, for example, has advantages and disadvantages in comparison with methods such as video recordings. The investigator might miss some navigation paths, resulting in an overestimation of task efficiency. Moreover, the observer may somehow influence a participant. Sitting just behind a participant may create a feeling of being rushed, which may lead to mistakes. A usability laboratory can facilitate in more rigorous data collection. Next, evaluating a system that was also developed by the evaluators could raise a conflict of interest. For example, interviewees knew that we were developing a new information tool. This knowledge could have elicited gratifying responses. We tried to overcome this by creating an open atmosphere, in which participants were encouraged to find usability problems. Another limitation is the think-aloud protocol applied in this study. Thinking aloud during usability tests can facilitate finding problems, as it reflects the actual use of a feature rather than the participant's judgment[19, 31]. Therefore, some authors have noticed that think-aloud interviews can impede the discovery of usability problems[32] and task performance[20].

In conclusion, our online Q&A tool is a promising new strategy for providing company workers with high quality information to answer OSH questions. The revised version, launched after this study, addressed the concerns and usability problems that were raised in the test and the interviews. Our tool seems to be particularly applicable to the provision of additional information on non-urgent health and safety topics, and can possibly improve contact between questioners from companies and OSH experts. Hosting and support by a trustworthy professional organisation, anonymity, timely answering and effective promotion campaigns were identified as important requirements for use.

This study indicates that Q&A network tools can be an interesting addition to existing information facilities in the field of OSH and in other healthcare fields that are looking for new strategies to answer questions from people in practice (workers, patients, or professionals) with high quality information. Nonetheless, this study was just a first step in a larger evaluation of the Q&A tool ArboAntwoord. In the near future, we will study the actual value of this tool within a given OSH knowledge infrastructure. We will focus on the use of the tool, the answer quality and the effects on information and knowledge dissemination in general. We recommend research on the use and effects of Q&A tools in different contexts.