Controlling Subjectivity When Business Jet Data Is Sparse
Low data does not mean low risk. A business jet operator may have only a small number of sectors, reports or maintenance findings in a review period, yet still need to make a defensible decision about whether a hazard is controlled. The danger is that a thin evidence base invites personal judgement to fill the gaps. One manager may call an issue isolated; another may see an emerging trend. Both may be acting in good faith, but an auditor will ask what method produced the conclusion, who applied it and what evidence was available at the time. The practical answer is not to manufacture statistical confidence. It is to separate what is known from what is assumed and to use a repeatable method for recording, reviewing and escalating limited information.
Low volume is not the same as low significance
Business jet operations naturally produce uneven data. A fleet may have a small number of aircraft, changing routes, different crews, and a mixture of owner, charter, positioning and training flights. A month with no fatigue report, unstable approach report or technical delay does not demonstrate that the underlying risk is absent. It may indicate that the exposure was low, that reporting is weak, or that the reporting threshold is unclear.
This distinction matters under the safety management framework established by ICAO Annex 19 and the applicable national CAA requirements. For an EASA operator, the management system requirements in Part-ORO provide the relevant structure for hazard identification, risk assessment, safety performance monitoring and records. Where an operator is subject to ORO.FTL, fatigue-related evidence also needs to be considered within the applicable flight-time limitation and fatigue risk processes. NCC and other non-commercial operations may be governed through a different national framework, so the accountable manager and nominated safety personnel must identify the rules that actually apply to the operation rather than importing an AOC process without checking its scope.
The first control is therefore classification. Record the number of operations or other relevant exposures, the number of reports received, the reporting source, the period covered and any known changes in activity. A zero count should remain a zero count. It should not be converted into “no risk” unless the operator has evidence that the exposure was monitored and the reporting system was functioning.
Replace impressions with a defined tally
Tallying is simple, but it becomes useful only when the unit being counted is defined in advance. “Safety events” is usually too broad. An operator might separately count unstable approach reports, runway excursions or rejected take-offs, fatigue reports, deferred defects, maintenance errors, ground-handling events, late technical releases and findings from internal audits. The relevant unit may be a flight sector, flight duty, aircraft day, maintenance task or audit sample. The choice should reflect the hazard being examined.
For each category, the safety manager should record the count, the exposure denominator where one is meaningful, and the source of the data. A tally of two fatigue reports over a quarter means something different if the operation conducted 40 flight duties rather than 400. At the same time, a denominator does not remove the need for judgement. Two reports involving the same crew pattern, aircraft or route may warrant immediate review even when the calculated rate appears small.
A workable low-data review can use four separate statements:
-
what was observed and how many times;
-
what exposure or population was reviewed;
-
what was not observed, including limitations in reporting or sampling;
-
what action, owner and review date follow from the evidence.
This structure prevents a common failure mode: a meeting record that says “no significant trend identified” without showing how the conclusion was reached. The decision may still be that no corrective action is required, but the record should show the tally, the period, the reviewer and the reason the remaining uncertainty is acceptable.
Make judgement visible and reviewable
Subjectivity cannot be removed from safety assessment. It can be controlled by making the judgement explicit. Before reviewing the data, define the conditions that would trigger escalation: a repeated event of the same type, a credible high-severity occurrence, a cluster involving one aircraft or crew pattern, a reporting gap, or a change in operation that invalidates the previous baseline. These triggers need not be universal numerical limits. They should be proportionate to the operator and documented in the management system procedure.
Use a short evidence register for each material risk. It should identify the hazard, the evidence reviewed, the date range, the data owner, the current assessment, the uncertainty and the next review date. If evidence comes from several sources, retain the source records: occurrence reports, flight-duty records, defect records, audit findings, training records, crew feedback and meeting minutes. A summary spreadsheet or SAFEJETS management-system record is useful only if the underlying evidence remains traceable and changes are controlled.
Review independence also matters. The person who owns an operational target should not be the only person deciding whether poor results indicate risk. In a small business jet organisation, complete independence may be impractical, but a second reviewer can still challenge the conclusion. The safety manager may prepare the tally, the head of flight operations may explain the exposure, and the accountable manager or nominated post holder may accept the residual risk and assign resources. The record should show these roles rather than attributing the decision to “management”.
Calibration is particularly important when several people assess similar events. Use previous examples, agreed severity descriptions and consistent escalation logic. Do not quietly change a category because the count looks inconvenient. If the classification changes, preserve the original entry, document the reason and explain whether earlier records were re-evaluated. An inspector may ask why the number in a dashboard differs from the number in occurrence reports; the answer should be available without reconstructing the history from email.
Low data should also produce a reporting-quality action where appropriate. If no reports are received but crews have limited access to reporting, managers do not explain the purpose of reports, or records are not closed with feedback, the finding is not “zero events”. It is an information-quality weakness. The corrective action may involve briefing, simplified reporting, protection from inappropriate attribution, or a focused sample of flight and maintenance records. The objective is to improve the evidence base, not to increase the number of reports artificially.
For audit purposes, retain the review agenda, the source-data extract or reference, the tally method, the assessment and the resulting action. Connect corrective actions to an owner and due date, then record effectiveness review. Under an SMS or management system, this chain is more defensible than a polished dashboard because it shows how the organisation identified uncertainty and managed it.
The practical discipline is straightforward: define what is being counted, preserve the denominator and the gaps, establish escalation conditions before the result is known, and require a second view of important judgements. Sparse data may justify a cautious conclusion, but it never justifies an undocumented one.