Elsevier

Safety Science

Volume 49, Issue 10, December 2011, Pages 1381-1387
Safety Science

The development of a prototype behavioral marker system for US Navy officers of the deck

https://doi.org/10.1016/j.ssci.2011.05.009Get rights and content

Abstract

The officer of the deck (OOD) of a US Navy ship is in charge of the safe and proper operation of the ship, and accountable to the commanding officer for every event that occurs during his or her OOD watch. This paper discusses the development of a prototype behavioral marker system to evaluate, and provide feedback on, the nontechnical (cognitive, social, and personal resource) skills of OODs.

An initial set of 17 categories of nontechnical skills were identified from a literature review. A focus group with four qualified OODs used the skills identified from the literature review to develop an initial taxonomy of five categories, each with two or three corresponding behavioral elements. This taxonomy was then used to classify 149 statements concerned with the nontechnical skills of OODs collected from 16 critical incident interviews. After three iterations of adaptations to the taxonomy, two independent raters were able to reach acceptable levels of reliability in using the taxonomy to classify the statements. Although further development work is required, it is suggested that the prototype behavioral marker system has implications for improving safety and performance on military and civilian ships.

Highlights

► Human error is a large contributor to maritime mishaps. ► Limited human factors research has been carried out into the nontechnical skill required by mariners. ► This paper describes the procedure used to develop a prototype behavioral marker system for officers of the deck. ► The behavioral marker system has implications for evaluation, and training, of the nontechnical skills of mariners.

Introduction

The officer of the deck (OOD) is a critical watch station on a US Navy ship. The OOD is accountable to the Captain for every event that occurs during his or her OOD watch. “As the Captain’s direct representative, the OOD is the only person on board who can make decisions that affect the safety of the ship and the lives of her crew” (Stavridis and Girrier, 2007, p. 3). There are 13 separate watch stations (individuals in the ship’s company responsible for carrying out particular tasks) that report directly to the OOD. When underway, the OOD is designated by the Captain to be in charge of the ship including its safe and proper operation (Chief of Naval Operations, 2005). “Nowhere in military or civilian life is there a parallel to the range and degree of responsibility that is placed in the hands of the OOD. As direct representative of the Captain, he or she acts with all the authority of command and, next to the Captain and the Executive Officer (XO), [the OOD] is the most important person on the ship” (Stavridis and Girrier, 2007, p. 1).

While technical skills and knowledge are obviously necessary for an effective OOD, they are not sufficient. The UK Marine Accident Investigation Branch (MAIB) stated that the majority of maritime accidents are attributed to human error (MAIB, 2000). Rothblum (2000) states that 75–96% of marine casualties are caused, at least in part, by human error. Similarly, Wagenaar and Groeneweg (1987) analyzed 100 Dutch shipping mishaps and found that in 96 of the mishaps those individuals involved could have prevented the mishap. In a recent study of mishaps involving Greek-flagged ships from 1995 to 2006, 57% of mishaps were attributed to human factors (Kokotos and Linardatos, 2011). Two examples of US Navy maritime mishaps in the public domain caused by poor nontechnical skills were the downing of an Iranian Airliner by the USS Vincennes in July 1988 (Klein, 1998), and the ramming of an anchored Spanish bulk carrier by the nuclear-powered aircraft carrier USS Dwight D. Eisenhower in August 1988 (National Transportation Safety Board, NTSB, 1990). It was determined that the probable causes of the ramming of the Spanish vessel were the delayed and insufficient action to correct the USS Eisenhower’s deviation from the intended track by the navigator and the OOD. The OOD did not adequately track the vessels movement (failure of situation awareness), and the commanding officer (CO) was not informed of a change of speed in a timely manner (failure of communication; NTSB, 1990).

Nontechnical skills are the “cognitive, social, and personal resource skills that complement technical skills, and contribute to safe and efficient task performance” (Flin et al., 2008, p. 1). Stavridis and Girrier (2007) state that the nontechnical skills of forehandedness, vigilance, judgment, intuition, and leadership are just as important to OODs as technical skills and knowledge. However, despite the recognized importance of nontechnical skills, there has not been any research conducted to identify the specific nontechnical skills required by the OOD. There are also no valid and reliable tools to evaluate and provide feedback on the nontechnical skills of OODs. The lack of a framework for evaluating nontechnical skills is not confined to the OOD watch station. There has been little human factors research conducted in the maritime industry more generally (Hetherington et al., 2006). The purpose of this paper is to carry out a task analysis to identify the nontechnical skills required by effective OODs, and develop a prototype behavioral marker system for evaluating these skills.

Section snippets

Behavioral markers

Behavioral markers are “observable, nontechnical behaviors that contribute to superior or substandard performance within a work environment” (Klampfer et al., 2001, p. 10). They are usually structured into a set of categories (e.g. co-operation, decision making, and situational awareness). Normally, these categories are then sub-divided into more specific nontechnical skills or elements.

The seminal research on behavioral markers comes from studies of civilian pilots carried out by Helmreich and

Literature review

The purpose of the literature review was to create an exhaustive list of nontechnical skills that have been shown to be necessary for effective performance in high risk domains, with a particular focus on any research that had been carried out in the maritime industry. Several electronic databases (PsychINFO, ScienceDirect, Web of Science, Google Scholar, and the Defence Technical Information Center) were used to identify research articles on human factors in the maritime industry by using the

Focus group

A group of SMEs identified which of the nontechnical skills in Table 1 were appropriate for assessing OODs. Once this had been carried out, the goal was for the SMEs to then agree upon an initial taxonomy of skills sub-divided into specific elements (the same structure as NOTECHS).

Critical incident interviews

The purpose of this stage of the research was to evaluate whether the taxonomy developed by the focus group (shown in Table 2) could be used to reliably classify the nontechnical skills used by OODs in actual scenarios. It was anticipated that this process would be iterative, and the raters would need to make changed to the taxonomy in order to achieve acceptable reliability. The scenarios were developed using the critical incident technique (CIT) to generate interview data for analysis. The

Discussion

The prototype Nontechnical Skills for Officers of the Deck (NTSOD) taxonomy, shown in Table 3, provides a structure for evaluating the nontechnical skills of OODs. However, further developmental work is required to ensure that the system can be reliably used by evaluators. Once a prototype behavioral marker system has been developed, the next stage is to get the anticipated end users to rate standardized video enactments of scenarios similar to those encountered in the actual environment (

Conclusion

Hetherington et al. (2006) stated that there are many gaps in the maritime human factors literature. We believe that the NTSOD taxonomy is a step towards filling these gaps. Although further research is required to establish the reliability and validity of the NTSOD taxonomy, it is one of the first research based behavioral markers systems developed for use in a maritime environment. It is suggested that the NTSOD taxonomy has implications for the teaching and evaluation of nontechnical skills

Acknowledgment

All opinions stated in this paper are those of the authors and do not necessarily represent the opinion or position of the US Navy, the Naval Postgraduate School, or the National University of Ireland, Galway.

References (38)

  • R. Flin et al.

    Safety at the Sharp End: Training Nontechnical Skills

    (2008)
  • Gatfield, D., 2005. Using simulation to determine a framework for the objective assessment of competence in maritime...
  • Helmreich, R., Wilhelm, J., Kello, J., Taggart, E., Butler, R., 1990. Reinforcing and Evaluating Crew Resource...
  • C.E. Hill et al.

    Consensual qualitative research: an update

    Journal of Counselling Psychology

    (2005)
  • C.E. Hill et al.

    A guide to conducting consensual qualitative research

    The Counseling Psychologist

    (1997)
  • E. Hollnagel

    Human Reliability Analysis: Context and Control

    (1993)
  • D. Jones et al.

    Sources of situation awareness errors in aviation

    Aviation Space, and Environmental Medicine

    (1996)
  • B. Kirwan et al.

    A Guide to Task Analysis

    (1992)
  • Klampfer, B., Flin, R., Helmreich, R., Häusler, R., Sexton, B., Fletcher, G., Field, P., Staender, S., Lauche, K.,...
  • Cited by (20)

    • IT-assisted equipment safety checks system to improve compliance: A case study at a distribution center

      2013, Safety Science
      Citation Excerpt :

      Many studies have found that cognitive feedback and preventive systems improved workers’ performance in terms of compliance to procedures, reduction of errors and improved safety (Bates and Gawande, 2003; Bates et al., 1998, 1999; Kontogiannis, 1999; Kontogiannis and Malakis, 2013; O’Connor and Long, 2011; Goomas and Yeow, 2010; Yeow and Goomas, in press). For example, Kontogiannis and Malakis (2013) and O’Connor and Long (2011) introduced the behavioral marker cognitive feedback system for air traffic controllers and US Navy Officers. Leveson (2004) proposed a cognitive feedback and prevention system for a chemical reactor plan.

    • Strategies in coping with complexity: Development of a behavioural marker system for air traffic controllers

      2013, Safety Science
      Citation Excerpt :

      Finally, we discuss the results in the context of adaptation of controller strategies and conclude with implications for error management training and design of ATC systems (Sections 6 and 7). The typical method for the initial development of behavioural marker systems is to carry out a literature review of domain specific research, followed by interviews and critical incident methods in order to extract strategies used by controllers in coping with complex scenarios (e.g., O’Connor and Long, 2008, study of navy deck officers). In the current study, the behavioural markers were extracted from an extensive literature review of the ATC domain and a previous study of enroute controllers (both experts and novices) in two training programmes (i.e., Malakis et al., 2010a, 2010b).

    • The Essential Handbook of Healthcare Simulation

      2023, The Essential Handbook of Healthcare Simulation
    View all citing articles on Scopus
    1

    Tel.: +1 757 450 7012.

    View full text