RSS

Statistical muddying

Wed, Oct 17, 2007 — David Evans

Briefs

Former FAA Administrator Marion Blakey said the next goal regarding safety is to further reduce the number of fatalities. There are a few considerations, however, to what she is saying. At the 6 September town hall meeting with FAA employees, Blakey said:

“We’ll be using a new performance metric for commercial air carrier safety – fatalities per 100 million persons on board … We aim to cut this risk in half … We are currently at 8.8 fatalities per 100 million persons on board, and will focus our efforts to get to 4.4 fatalities by 2025.”

All well and good, but few people have a feel for a number as big as 100 million, so whether the fatality rate is 8.8 or 4.4 of 100 million really is an abstraction. Most people have their brains full with the notion of a mere 100,000 at a football game, so a base number that is 1,000 times greater basically has no practical meaning.

For another thing, whole numbers are preferable to eight-tenths or four-tenths of a person killed.

For yet another thing, achieving a rate of 4.4 persons killed per 100 million by 2025 is difficult to relate to the past goal of reducing the number of fatal airline accidents by 80% over the past decade. The past goal spans 10 years; the future goal covers a period of 18 years – the two are not comparable.

There is no mention by Blakey of a 3-year rolling average, or the fact that the FAA has kept book for the past decade by fiscal year, not calendar year. Fiscal year accords with the FAA’s budget cycle, but recording accidents from October through September, rather than January through December, just makes it difficult to compare the FAA rates to other databases.

Case in point, the accident and incident data maintained by the National Transportation Safety Board (NTSB). It is more detailed (and sensible) than the way the FAA keeps score on safety. For airlines, the NTSB data base tracks non-fatal as well as fatal accidents, thereby covering significant damage to aircraft and injuries. The database also tracks accidents annually, per departures, per flight hours, and per miles flown (see Figure A ).

It would be sensible if the FAA were to emulate the NTSB’s bookkeeping in its goal setting.

The same observation applies to general aviation (GA). Blakey has said, “There have been 284 fatal accidents, versus a not-to-exceed ceiling of 331 for the year.” The first problem is the dubious provenance of this not-to-exceed number. The original goal was to eliminate a year’s worth of fatal accidents. Tom McSweeny, then the FAA head of regulation and certification, said in March 2001 that “if we did nothing” there would be 437 GA accidents in 2007, and the goal was to cut that projection by 350 accidents and save more than 600 lives.

None of the FAA’s numbers can be related to the NTSB accident database for GA (see Figure B).

One thing is clear: the GA accident rate, at least in terms of gross numbers, is entirely too high. Blakey’s allusion to 284 accidents last year, while lower than the number shown on the NTSB database, reduced GA accidents from one every 29 hours implied in the not-to-exceed goal to one fatal accident every 31 hours. Compare this rate to that of two airline fatal accident rates per year, now achieve, and it is evident that reducing the GA fatal accident rate to even just one a week would be a major improvement.\

Irrespective of goals, the FAA should set its safety improvement goals in accordance with the NTSB bookkeeping of accidents. In this regard, setting a goal of reducing fractional fatalities per 100 million persons on board is an abstraction times two: the NTSB database counts whole deaths, and it does so in increments of 100,000 flight hours.

Figure A

Year Accidents Fatalities Accidents
per 100,000
Flight Hours
Accidents
per 1,000,000
Miles Flown
Accidents per
100,000 Departures
All Fatal Total Aboard All Fatal All Fatal All Fatal
1987 32 4 231 229 0.306 0.030 0.0075 0.0007 0.425 0.041
1988 29 3 285 274 0.266 0.019 0.0066 0.0005 0.381 0.027
1989 23 8 131 130 0.217 0.075 0.0053 0.0018 0.316 0.110
1990 21 6 39 12 0.182 0.052 0.0045 0.0013 0.269 0.077
1991 22 4 62 49 0.198 0.036 0.0048 0.0009 0.293 0.053
1992 16 4 33 31 0.136 0.034 0.0034 0.0008 0.213 0.053
1993 22 1 1 0 0.184 0.008 0.0045 0.0002 0.285 0.013
1994 18 4 239 237 0.138 0.033 0.0033 0.0008 0.217 0.051
1995 30 1 160 160 0.235 0.008 0.0056 0.0002 0.370 0.012
1996 31 3 342 342 0.239 0.023 0.0057 0.0006 0.395 0.038
1997 43 3 3 2 0.285 0.020 0.0068 0.0005 0.433 0.030
1998 42 1 1 0 0.264 0.006 0.0066 0.0002 0.399 0.009
1999 40 2 12 11 0.240 0.012 0.0060 0.0003 0.368 0.018
2000 49 2 89 89 0.280 0.011 0.0069 0.0003 0.443 0.018
2001 41 6 531 525 0.216 0.012 0.0053 0.0003 0.348 0.019
2002 35 0 0 0 0.209 0.0051 0.341
2003 51 2 22 21 0.302 0.012 0.0073 0.0003 0.499 0.020
2004 24 1 13 13 0.132 0.005 0.0032 0.0001 0.223 0.009
2005 34 3 22 20 0.182 0.016 0.0043 0.0004 0.312 0.027
2006 25 2 50 49 0.132 0.011 0.0031 0.0003 0.223 0.018
Source: www.ntsb.gov/aviation/Table6.htm

Figure B

Year Accidents Fatalities Accidents
per 100,000
Flight Hours
All Fatal Total Aboard Flight Hours All Fatal
1987 2,494 446 837 822 26,972,000 9.18 1.63
1988 2,388 460 797 792 27,446,000 8.65 1.66
1989 2,242 432 769 766 27,920,000 7.97 1.52
1990 2,242 444 770 765 28,510,000 7.85 1.55
1991 2,197 439 800 786 27,678,000 7.91 1.57
1992 2,111 451 867 865 24,780,000 8.51 1.82
1993 2,064 401 744 740 22,796,000 9.03 1.74
1994 2,021 404 730 723 22,235,000 9.08 1.81
1995 2,056 413 735 728 24,906,000 8.21 1.63
1996 1,908 361 636 619 24,881,000 7.65 1.45
1997 1,844 350 631 625 25,591,000 7.19 1.36
1998 1,905 365 625 619 25,518,000 7.44 1.41
1999 1,905 340 619 615 29,246,000 6.50 1.16
2000 1,837 345 596 585 27,838,000 6.57 1.21
2001 1,727 325 562 558 25,431,000 6.78 1.27
2002 1,715 345 581 575 25,545,000 6.69 1.33
2003 1,740 352 633 630 25,998,000 6.68 1.34
2004 1,619 314 559 559 24,888,000 6.49 1.26
2005 1,669 321 563 558 23,168,000 7.20 1.38
2006 1,515 303 698 538 22,800,000 6.64 1.32
The 698 total fatalities in 2006 includes the 154 persons killed aboard a foreign registered Boeing 737 aircraft operated by Gol Airlines when it collided with an Embraer Legacy 600 business jet over the Brazilian Amazon jungle. Source: www.ntsb.gov/aviation/Table10.htm

Comments are closed.

Nolan Law Group