Ten Operating Characteristics

NOSS is defined by ten operating characteristics that distinguish it from other observational methodologies. These characteristics are based on the observational method literature and years of experience in conducting field studies. These ten characteristics are designed to foster trust in the NOSS process and promote the consistent handling of NOSS data.

Over-the-shoulder observations during normal shifts

NOSS data is collected during unobtrusive observations of normal shifts. NOSS aims to encourage controllers to behave as normally as possible during observations, so proficiency checks and OJTI are excluded from NOSS as controllers may display “angel performance” while being formally evaluated. Observations are conducted by specially trained peer controllers who do not wield punitive powers over observed controllers – allowing observed controllers to perform as normal without concern.

Anonymous, confidential, and non-punitive safety data collection

Trust and confidentiality are crucial to the integrity of NOSS. Numerous steps are taken to protect confidentiality and bolster trust. NOSS does not record names, dates, or other potentially identifying information. Additionally, observer’s identities are not linked to individual observations providing another layer of confidentiality.

Voluntary participation

Controllers may refuse to be observed, which differentiates NOSS observations from proficiency checks. This is an important step in countering suspicion and gaining trust. Rather than NOSS “being done” to controllers, it is participatory, which can enhance trust, data quality and controller support of the NOSS process.

Trusted and trained observers

Observers are critical to NOSS success. Observers must be trusted and respected by controllers, or they will refuse to be observed or demonstrate “angel performance.” Observers must be properly trained to adhere to NOSS protocols and record quality data.

Most NOSS observers are operational controllers from the units conducting NOSS, which helps those units to feel a partner in the process. NOSS observer teams are sometimes augmented with observers from (similar) external units or various departments that support the operation (training, safety, procedures & standards, etc.)

Joint management / association support

Joint management and controllers’ association support can greatly enhance controller acceptance of NOSS, thus increasing trust and the quality of the data collected as well as yield benefits during the post-NOSS safety change process.

NOSS has been cited as a jointly supported activity at several ANSPs that improved relations between management and front-line staff.

Systematic data collection instrument based on Threat and Error Management (TEM)

Threat and Error Management (TEM) guides NOSS data collection, with the instrument structured to capture objective observer descriptions while minimizing observer judgments.

Trusted data collection site

NOSS data must be sent to a sort trusted by management, observers and observed controllers to prevent concerns for data being misused. The more comfortable observers and observed controllers are with confidentiality and data security, the more comfortable they will be to demonstrate and record “natural performance,” thus increasing the quality of the data collected.

When working with the NOSS Collaborative to implement NOSS, all observations are typically sent directly to the NOSS Collaborative.

Data verification process

After observations have been conducted, a crucial quality control step called data verification is conducted. Data verification has multiple purposes. It’s primary purpose is to ensure data quality – operational experts from participating units vet the data to ensure it is fit for inclusion in the analysis, is properly coded and consistently coded. Expert discussion of observed events also allows for additional insight to be captured to inform the observations. No data analysis is done until data verification is complete.

Data derived targets for safety enhancement

NOSS findings are summarized in a comprehensive report with a mixture of quantitative and qualitative information. Various statistical measure are used to summarize and benchmark findings with de-identified units in the NOSS Collaborative Archive. Statistical summaries are incomplete, however, as they lack context about the specifics of what is happening, and why.

Feedback of results to controllers

Once the ANSP / Unit has had an opportunity to digest the NOSS report, findings should be made available to controllers – in some form. ANSPs have taken different approaches in sharing findings with staff, from making the entire report available to sharing key findings and organizational plans to address findings.