Human Cognitive Workload Assessment Tool

Project Summary

Overview of project
What is cognitive workload?
Review of the current literature on cognitive workload
Identification of factors that influence workload
Development of the tool using Influence Diagrams

In 2006, Human Reliability worked on behalf of the Maritime and Coastguard Agency to develop a software tool for the assessment of human cognitive workload levels, intended for use in the maritime industry. The tool, called CLIMATE, can be used to analyse specific work scenarios and report whether they fall within safe working limits.

Click here for more information about CLIMATE.

The overall aims of this project were:

• Review current research into safe maximum and minimum human cognitive workload capabilities.
  
• Identify the safe maximum and minimum human cognitive workload levels for the maritime industry and, if necessary, for different trades or conditions of work within it.
  
• Develop a robust tool that can effectively and efficiently assess human cognitive workload levels.
• Test the tool using examples of rosters/shft patterns from the maritime industry.

The full report will soon be available on the MCA's website.

What is Cognitive Workload?

Cognitive workload describes the level of mental resources required of a person at any one time. This affects their ability to process information, to react to their surroundings and to make decisions. Reducing these abilities can increase the likelihood of an accident occurring to that seafarer or their vessel. It can be thought of as a scale with two opposite ends:

• Overload – “too much information”. For example, when berthing a vessel in an unfamiliar port in poor weather, the level of information processing and decision making required of a crewmembers will be very high.
• Underload – “daydreaming”. For example, when keeping watch at night in calm seas with no nearby traffic and all systems working normally, it can be difficult to maintain a satisfactory level of vigilance to respond to an unexpected event that may arise.

The degree of underload or overload will vary depending on any mismatch between the demands placed on a person and the mental capacity that they have available to meet those demands.

Review of the current literature on cognitive workload

The first part of this study focused on the current literature on Cognitive Mental Workload (CMWL). Relatively few treatments of the topic have focused upon consideration of workload error at both the upper and lower limits. The mental workload literature has tended to be mostly concerned with performance problems associated with tasks high in explicit demand (i.e., the overload or cognitive strain scenario). There is, however, a separate major strand of work aimed at exploring/modelling errors at lower levels of the task demand spectrum.

In this area, investigators have focused primarily on the performance of tasks requiring vigilance (e.g., target detection, usually in unstimulating situations such as watchkeeping). Predictable performance shortfalls occurring during a vigil are known collectively as the Vigilance Decrement. Vigilance studies have been carried out extensively in maritime operations where detection of infrequently occurring targets has historically been both problematic and a common feature of the work situation.

The overall theoretical position that emerges from the literature is that Cognitive mental Workload (CMWL) arises primarily from a mismatch between the demands of the task itself, (e.g. e.g. judgements about manoeuvring and monitoring the position of a ship during berthing) and the mental resources available to meet these demands. These include the skills and training of the OOW, the supporting bridge team, and technical support systems such as radars and GPS that supplement these human resources, if well designed.

The second part of the review focussed on workload assessment techniques. Five main approaches to workload assessment were identified:

• Subjective methods
• Task performance methods
• Physiological monitoring
• Task loading
• Influence Diagram Evaluation System (IDEAS)

The first four of these approaches have been used to develop a wide range of tools for workload assessment, mainly in industries traditionally defined as high risk such as defence, road transportation, railways, aerospace, process control, and power generation. The choice of method is usually based on consideration of the special requirements of the industry concerned. For example, workload assessment for drivers of road vehicles is typically based on performance of a range of secondary test tasks completed concurrently with performance of the primary driving task. Conversely, use of physiological monitoring techniques is frequently used in the defence industry, which has a ready supply of personnel, equipment and high fidelity simulators. Consequently, staff can be recruited to participate in realistic missions using advanced system simulators with performance being monitored using a variety of sensor equipment.

In addition to these specific CMWL assessment methodologies, a more general framework for human performance modelling was also included, which appeared promising as a means for basing the workload prediction tool on the knowledge and experience of mariners.

IDEAS is an application framework, which allows the insights from theoretical research and from people with extensive practical experience to be combined in the form of a simple graphical model of workload. This technique was selected for use in this study.

Identification of factors that influence workload

The CMWL literature review revealed a number of generic psychological factors likely to influence task performance given the presence or absence of task load. In developing final versions of the CMWL models, these factors were supplemented by known performance influencing factors (PIFs) associated with reliability evaluations and other factors specific to shipping operations.

Known PIFs in high workload conditions:

• Competition for resources
• Opportunity for adoption of workload management techniques
• Nature of the task demands

Known PIFs in low workload conditions:

• Audible alarms
• Signal salience
• Signal uncertainty
• Foreground/background effects
• Nature of the task demands

A number of marine accident databases were also examined to obtain an overview of the types of accidents in which CMWL overload and underload are contributory factors, and to identify which commonly occuring PIFs may have contirbuted to workload in these situations.

The PIFs identified were:

• Time of the accident
• The stage of the individual's shift
• Number of individuals on watch
• Type of vessel
• Technology used
• Weather conditions
• The area of the incident
• Fatigue
• Environment

The review of marine accident databases provided some useful insights into the types of scenario that the CMWL measurement tool needed to address and some of the PIFs that affect the level of workload. These and other PIFs also contribute to the likelihood of an error leading to an accident. Examples of incidents where underload, overload and the switch from underload to potentially overloaded situations were identified. However, in common with many accident investigation databases, the nature of the investigation process used tended to focus on the ‘what happened’ rather than the ‘why’ of accident causation, and hence the insights into specifically workload related factors are quite limited. Nevertheless, the more general recurrent PIFs that have been identified are useful if the CMWL measurement tool is to be linked with a marine error prediction process.

Development of the tool using influence diagrams

Using the insights gained from the research conducted earlier in the project, such as the analysis of shipping incidents, and the review of mental workload literature, a framework model of the factors influencing CMWL was developed which expressed workload as a function of measurable characteristics of tasks, personnel and the context within which marine operations are performed. Separate models were developed for underload and overload situations, since the underlying processes that give rise to these states are very different, based on the research literature considered in the survey. In order to populate the model, data were drawn from the research literature, and structured workshops with subject matter experts from the marine industry.

The framework model, including the factors influencing CMWL, was expressed in the form of a family of Influence Diagrams, using an already available Influence Diagram modeling software tool called IDEAS. This technique provides a modeling environment which allows the insights arising from theory based approaches, together with the knowledge and experience of mariners, to be combined in a form suitable for assessment. They also provide a method for allowing workload assessment to be linked to externally measurable data such as incident reports to allow verification of the predictions of the tool.

This framework model was used as the starting point (or seed model) for a series of interactive sessions with subject matter experts from the commercial marine industry, including Masters, First and Second Officers, and recently trained personnel. These sessions were used to develop workload models for different domains of shipping activities from high speed ferries, coasters and deep sea operations such as tankers. Both overload and underload models were developed. These models were then validated by using them to assess scenarios known to the workshop participants

The main benefit of this approach was that it was pragmatic, evidence based, and did not depend on the accuracy of specific theoretical approaches in order to produce a workable CMWL tool. The use of the IDEAS modeling tool allowed the project to build on an existing tried and tested modeling environment that has been successfully utilized in a wide range of applications over the past 10 years. This approach also allowed more of the software resources to be utilized for the development of the interface and user facilities.

Click here for more information about Influence Diagrams.

Back to the top