Subjective Evaluation of Seafarers’ Situational Awareness on Collision Avoidance

Objectives: This study aims to investigate the situational awareness capabilities of seafarers which is the weakness of the safety management system studies conducted under the human element researches in maritime literature. Methods/ Statistical Analysis: The SART multi-dimensional, self-rating, post-trial situational awareness measurement technique was used to assess the situational awareness on collision avoidance of the seafarers who have different ages and experience levels. The results were analyzed usingdescriptive statistics, one-way ANOVA and post-hoc tests (Tukey, LSD and Bonferroni). Findings: Results indicate that a gradual decline in mean SART scores of the seafarer from oceangoing watchkeeping officers to oceangoing masters is clearly detected. Although this decline is observed in the all SART main dimensions (Understanding + Attentional Demand + Attentional Supply), there is no statistically significant difference among the situational awareness levels of seafarers in SART generic analysis. Application/Improvements: The findings of this study suggest that, recruiting seafarers by examining the requirements ofthe situational awareness will be a critical stepto minimizeaccidents on board.


Introduction
Since maritime transportation is safer, more economical and environmentally friendly than other transportation systems, it constitutes about ninety percent of the global transportation volume and shows an increasing trend 1 . Today's ship systems are technologically advanced and highly reliable. However, ship structure and system reliability constitute a relatively small part of the ship safety equation and optimization of only these factors provides limited development 2 . Major operations on board such as navigation, cargo handling, ship management and maintenance require direct human element inputs. The statistics in the literature explain that the human factor is one of the root or most important causes of marine accidents that cause great damage to nature, economy and maritime sector 3 .
Human factor in maritime domain, is a multidimensional and complex structure that spans the entire operational process of the ship including design and construction, ship management companies, regulations, shipyards, suppliers, ports and all other relevant units 4 . Within this structure, it is a known fact that people can make mistakes as a functional system element. In this sense, the human factor based error can be caused by the limitations and delays in the cognitive abilities that enable individuals to take input from the external environment and develop behavior such as problem solving, learning, planning 5 .
The concept of situational awareness is defined as a cognitive structure, starting with perception -ending with decision making and covering many cognitive processes such as attention, working memory, spatial processing, learning 6 . Studies in the literature mainly focus on the causal relationships between individuals' situational awareness levels and task performances, and define situational awareness as an indicator of performance. High situational awareness is explained as part of successful performance, while losses in situational awareness often represent one of the reason for unsuccessful performance 7 .
Although situational awareness researches are predominantly carried out in the aviation field where basic models of the concept are reveled, it is also quite important for other areas where human-machine interaction is intense and have similar operational processes. The studies carried out in the maritime domain in the last two decades draw attention to the potential importance of situational awareness in terms of safety and performance 8 .
The main purpose of this study is to make an overall and subjective situational awareness assessment for seafarers based on the three-level situational awareness model of Endsley 9 , detailed in the literature review section. For this purpose, the SART multidimensional situational awareness measurement technique was used to assess situational awareness of seafarers with different levels of experience. During the analysis, the seafarers consisting of oceangoing masters, oceangoing chief officers and oceangoing watchkeeping officers rated the situational awareness dimensions contained within the SART form according to their perceived performance based on the collision avoidance task they performed in the previous step. This paper is divided into five chapters. In the introduction and literature review parts, the human element and situational awareness literature that motivate us for this research are given. The third chapter is concerned with the characteristics of the participants, data collection process and statistical method. The fourth section introduces the findings of the research. And lastly, a brief critique of the findings is presented.

Situational Awareness Literature
The concept of situational awareness was first defined as "the importance of gaining an awareness of the enemy before the enemy gained a similar awareness, and devised methods for accomplishing this" during World War I. Within the following time; although the concept of situational awareness could not find much space in the academic literature, it has become a very popular topic since the late 1980s. In this process; while some researchers criticized the concept of situational awareness being incoherent, subjective, and intuitional definition 10 , other researchers responded to these claims, arguing that this concept was particularly important for operational tasks 11 .
Although the foundations of situational awareness concept are based on studies in aviation domain, it is particularly important for other areas where have simultaneous monitoring of multiple targets by operators, presence of different tasks to attract the attention of operators and under the time stress 12 .
Today, many situational awareness definitions and models have been developed within the scope of human factor studies. Among the proposed models, the most highlighted in the academic literature is the three-level situational awareness model of Endsley, which is shown in the Figure 1. This model based on the information processing approach and many cognitive processes consists of three levels: SA1-Perception, SA2-Comprehension, SA3-Projection9. It is the lowest level of situational awareness and at this stage the elements in the surrounding environment are processed by the operator. Interpretation of data is not carried out at this level. Level-1 is mainly a data collection process and the elements in the environment are defined independently. At this stage; if new elements appear in the environment, the operator can verify this data.
SA2-Comprehension: Comprehending of the current situation. At this level, the data collected at the previous level is separated to comprehend the situation. A holis-tic view of the surrounding environment is provided. Endsley states that the degree of comprehending is an indication of the individual's expertise and experience. An individual with less expertise and experience than his colleagues may achieve a lower level of comprehension (SA2), although he has achieved the same level of perception (SA1) as his colleagues.
SA3-Projection: Projection of future situation. It is the highest level of situational awareness and is associated with the ability to predict the future of elements in the environment. The accuracy of the projection is directly proportional to the quality of the lower levels (SA1 and SA2). This level enables for the individual to gain time to solve conflicting information and to prepare an action plan for their goals9.
Each of these levels in the three-level situational awareness model includes identifiable cognitive processes and associated performance losses. In this model, situational awareness is a separate structure from decision making and performance. However, situational awareness is just ahead of decision making and is an important component of the dynamic decision-making process. The concept of situational awareness defined in the model includes only the part of the individual's knowledge that is relevant to the state of a dynamic environment. Although expertise, experience and system rules can influence the development of situational awareness, they are static sources of information outside this concept. For example; prejudices based on previous experience affect the process of situational awareness by directing operator's attention 13 .
Lack of consensus on the definition and theoretical structure of situational awareness also affects the stage of measurement and evaluation of this concept. Most of the situational awareness measurement techniques in the literature have been developed based on a theoretical model. This makes the selection of an appropriate situational awareness measurement approach difficult. In the human factor and ergonomics literature, there are about thirty different approaches designed to measure situational awareness 14 .

Method
The SART (Situational Awareness Rating Technique) multidimensional situational awareness measurement technique was used to assess subjective situational awareness of seafarers. For this purpose, the requirements of the collision avoidance were analyzed with the support of specialists and a scenario through the İstanbul Strait was conducted in the İTÜ Maritime Faculty CBT (Computer Based Training) laboratory about twenty minutes. The seafarers rated the situational awareness dimensions in the form of SART according to their perceived task performance based on the mission they performed. The obtained results were brought together in the relevant formula and a general and subjective situational awareness assessment was carried out for seafarers.

Participants
Stratified random sampling approach was used to represent the seafarers who constitute the universe of our study. Research sample consists of 140 seafarers in three strata: 37 oceangoing masters (aged between 33 and 54), 47 oceangoing chief officers (aged between 28 and 39) and oceangoing watchkeeping officers (aged between 22 and 31). These seafarers have same education level (graduate) and actively work at sea. All participants are volunteers and received no immediate benefits from participating in this study.

Data Collection and Procedure
SART is a post-trial, multi-dimensional, self-rating situational awareness scale designed by Taylor to evaluate the situational awareness of pilots 15 . The SART scale defines the concept of situational awareness in ten dimensions (instability of the situation, complexity of the situation, variability of the situation, division of attention, concentration, spare mental capacity, arousal, information quality, information quantity, familiarity). Each of these ten dimensions is assessed by individuals using a sevenpoint rating scale (1: low, 7: high) according to their perceived task performance. After the measurement, the scores are combined to calculate the level of situational awareness of the participants in the relevant formula. There is also a faster version of the SART technique, known as 3D-SART, which limits the ten dimensions of situational awareness to three main dimensions (Attentional demand, Attentional supply, Understanding). Table 1 shows the dimensions of the situational awareness in the SART scale and the explanations of these dimensions 16 .
There are four basic steps that should be followed to obtain effective results on the situational awareness of individuals in SART scale. The first step is to determine the purpose of the study and to define the task for this purpose. In this study; in order to evaluate the situational awareness of seafarers on collision avoidance, an Istanbul Strait transition simulation task has been defined shown in Figure 2 in the İTÜ Maritime Faculty CBT (Computer Based Training) laboratory. The second step consists of selecting and informing the participants. Before performing the task; all participants were informed about the aim of the study, the concept of situational awareness, SART method and dimensions. The third step involves performing the task and filling the SART form. After the task is completed, SART form was given to the participants and they were provided to rate for each dimension based on their perceived task performance. In this process, participants should be allowed to ask questions to understand better the SART dimensions. However, all guidance and interventions that may affect the participants' assessments should be avoided. The final step involves the calculation of each participant's SART score using the following formula.
Situational Awareness = Understanding -(Attentional Demand -Attentional Supply) The SART scale has the advantages of subjective situational awareness assessment techniques such as rapid implementation, low cost and easy management. On the other hand, SART has high ecological validity as its dimensions have been developed directly from the operational air crew. The ecological validity of methods developed in laboratory or artificial environment is generally low 17 . However, although SART provides effective information on individuals' situational awareness, they may be affected by performance. In other words; when the task is successfully carried out and a positive result is achieved, a person can evaluate its situational awareness at a higher rate. In addition, it is directly affected by the memory weaknesses of individuals as it is a post-trial technique 18 .
It is a proven fact that there is no correlation between subjective and objective measurement techniques of situational awareness. SART, which is one of the subjective measurement techniques of situational awareness, has high correlation with individual's confidence and performance. However, this fact does not make subjective

Statistical Analyses
In this study, descriptive statistics, one-wayANOVA andpost-hoc tests (Tukey, LSD and Bonferroni)are used to analyze the data obtained from seafarers within different ages and experience levels.All results provide the normal distribution condition required for statistical parametric analysis (skewness and kurtosis coefficients between -1 and +1) 20 . According to variance homogeneity test (Levene's test), all comparison groups have equal variances (p> ,05). In addition, missing data are not detected in our study.

Results
SART results obtained from seafarers were analyzed separately for each SART dimension (Attentional demand, Attentional supply, Understanding) and a generic situational awareness measurement.

SART-Understanding Dimension Analysis Results
The understanding, which is one of the main dimension of the SART technique consists of the sum of three sub-dimensions (Information quantity + Information quality + Familiarity). Each of the sub-dimensions in the SART technique is evaluated by using a seven-point rating scale (1: low, 7: high) according to the participants' perceived task performance.Descriptive statistics of SART-understanding dimension based on the rating of seafarers for each separate group is shown in Table 2.
According to Table 2, the oceangoing masters have the highest mean SART-understanding score (16.34+1.72) than the other groups. Almost linear increase is observed in the mean score from oceangoing watch keeping officers to oceangoing masters. ANOVA test results indicate that, there is a significant difference in terms of amount of information received and understood, degree of goodness of information, degree of acquaintance with situation experience among our groups (F2,137: 12.53, p<0.05, η 2 :0.15).The post-hoc tests also point out that only oceangoing wacthkeeping officers (Xowo= 16.34) statistically

SART-Attentional Demand Dimension Analysis Results
The attentional demand main dimension of SART requires the evaluation of sub-dimensions together (Variability + Instability + Complexity). Each of the subdimensions is assessed by using a seven-point rating scale (1: low, 7: high) according to the participants' perceived task performance. Descriptive statistics of SART-attentional demand dimension for each comparison group is shown in Table 3. As shown in Table 3, oceangoing wacthkeeping officers have an observable mean SART-attentional demand score (16.71+1.88) compared to oceangoing chief officers and oceangoing masters. According to one-way ANOVA test results, statistically significant difference in points of related SART dimension which are probability of situation to change suddenly, number of variables requiring attention and degree of complication of situation is identified among our group means (F2,137: 7.95, p<.05, η2:0.10). SART-attentional demand dimension has the lowest effect size (η 2 : 0.10) among all the SART dimensions. The post-hoc tests show that only oceangoing wacthkeeping officers (Xowo= 16.71) are separated from other comparison groups (p<0.05). The mean SART -attention demand score of all seafarers are measured as (XSUM = 15.95).

SART-Attentional Supply Dimension Analysis Results
The attentional supply which is the final main dimension of the SART technique consists of a combination of four sub-dimensions (Arousal + Spare mental capacity + Concentration + Division of attention).In this sense, each of the related sub-dimensions in the SART scale is evaluated by using a seven-point rating scale to measure the situational awareness of the seafarers towards the dimension of attentional supply, Descriptive statistics of SART-attentional supply dimension for each comparison group is shown in Table 4.  As seen in Table 4, gradual decline in mean SARTattentional supply scores of the seafarers from oceangoing watchkeeping officers (23.30+1.83) to oceangoing masters (20.65+2.13) is clearly observed. According to one-way ANOVA test results, the mean SART-attentional supply scores of the comparison groups show statistically significant difference in points of degree of readiness of the individual to the activity, level that one's thoughts are brought to bear on the situation, the amount of mental ability an individual has for new variables and the amount of division of attention in the situation (F2,137: 20.83, p<0.05, η2:0.23). The post-hoc tests also indicate that only oceangoing wacthkeeping officers (Xowo= 23.30) statistically differ from other comparison groups in terms of the requirements of the attentional supply dimension of the SART scale (p<0.05).

SART-Generic Analysis Results
Generic evaluation of the SART technique consists of the sum of three main dimensions (Understanding + Attentional Demand + Attentional Supply). For this purpose, the seafarers rated the situational awareness dimensions included in the SART scale considering the dynamics of the given collision avoidance task. Descriptive statistics of SART generic situational awareness scores for comparison groups composed of seafarers is demonstrated in Table 5.
As shown in Table 5, there is a trend of increasing in average situational awareness scores in parallel with the competency of seafarers (XOWO=22.94, XOCO=23.34, XM=23.38). However, according to one-way ANOVA test results, this increase is not statistically significant (p>0.05). Results show that, there is no difference among situational awareness scoresof the oceangoing watchkeeping officers, oceangoing chief officers and oceangoing masters based on the self-evaluations.

Conclusion and Discussion
In this study, the situational awareness of the seafarers who have different ages and experience levels is evaluated by using SART technique. Results of the analysis obtained from the scenario task designed by considering collision avoidance requirements show that, a gradual decline in mean SART scores of the seafarers from oceangoing watchkeeping officers to oceangoing masters is clearly observed. Although this decline was observed in the all SART main dimensions (Understanding + Attentional Demand + Attentional Supply), there was no statistically significant difference among the situational awareness levels of seafarers in SART-generic analysis. The results of this research support the idea that, recruiting appropriate seafarers by examining the requirements of maritime operations in terms of situational awareness will is a critical step to reduce losses in the age-related cognitive decline process.