Federal Update
GAO report: CSA lacks ability to determine crash risk, disproportionately affects small carriers

By Jami Jones, managing editor

If the GAO report on CSA were a kid’s report card, there would be some unhappy parents.

The Government Accountability Office released its report on the Federal Motor Carrier Safety Administration’s Compliance, Safety, Accountability compliance measurement program in February. And the review is anything but glowing.

The compliance measurement and enforcement program is riddled with problems that are preventing the agency from effectively and accurately identifying motor carriers most likely to pose a crash risk on the highways, according to the report.

Small carriers ‘disproportionately’ affected
CSA collects roadside inspection violation and crash data to assess which motor carriers are most likely to pose a crash risk on the highways. Motor carriers are then ranked in Behavioral Analysis and Safety Improvement Categories – dubbed “BASICs” by FMCSA.

The GAO was highly critical of the amount of relevant data collected and in cases where only a few inspection reports exist on a motor carrier, how that lack of relevant data can circumvent the purpose of the program.

Most motor carriers lack sufficient safety performance information to ensure that FMCSA can reliably compare them to other carriers, according to the report. That puts the agency in a situation where carriers are mislabeled as a crash risk by their CSA compliance rankings.

“Carriers with few inspections or vehicles will potentially have estimated violation rates that are artificially high or low and thus not sufficiently precise for comparison across carriers,” the report states.

The reality posed for the agency is the likelihood that many scores do not represent an accurate or precise safety assessment for a motor carrier.

“For example, for a carrier with five inspections, a single violation could increase that carrier’s violation rate 20 times more than it would for a carrier with 100 inspections,” the GAO report explains. “This sensitivity can result in artificially high or low estimated violation rates that are potentially imprecise for carriers with few violations.”

FMCSA attempts to compensate for the variances in inspection levels by grouping motor carriers with similar inspection rates within the BASICs – which are categories of similar regulations that FMCSA attempts to score compliance. Carriers are then compared against other carriers in the same group and ranked based on the violation data of the other carriers.

The GAO did not see this as an effective way to accurately assess a carrier’s tendency toward crash risk.

“For example, FMCSA’s Hours of Service Compliance BASIC has five safety groups. The group of carriers with the fewest number of inspections in each safety event group tends to have a higher percentage of carriers identified as above the intervention threshold than the group of carriers with a greater number of inspections,” the report states.

The GAO report concludes that this suggests that the methodology does not adequately take into account exposure – be it exposure to frequency of inspection, locations where carriers operate, etc. But rather it is “systematically assigning higher scores for carriers with fewer inspections.”

That grouping of motor carriers, GAO states, “demonstrates how imprecision disproportionately affects small carriers.”

“This report states everything we have been saying all along,” said OOIDA Executive Vice President Todd Spencer. “The system has fundamental flaws and this report clearly points this out.”

Too much data not relevant to crash risk
Currently, the FMCSA collects violation data on 750 different regulations. The GAO analysis of that data determined that 593 of those regulations were violated by less than 1 percent of the carriers.

The GAO drilled down more and found that in at least half the tests conducted, violations of 13 of the regulations consistently had some association with crash risk. Only two violations – failing to wear a seatbelt and speeding – had sufficient data to consistently establish a substantial and statistically reliable relationship with crash risk across all of the GAO tests.

“First, we found that the majority of regulations used to calculate (CSA) scores are not violated often enough to strongly associate them with crash risk for individual carriers. Second, for most carriers, FMCSA lacks sufficient safety performance information to ensure that FMCSA can reliably compare them with other carriers,” the report states.

FMCSA attempts to compensate for infrequency of violations by grouping violation data together and attempting to establish crash risk of a motor carrier based on that grouped data. The GAO said this method fell short of establishing true crash risk.

“FMCSA has not demonstrated relationships between groups of violations and the risk that an individual motor carrier will crash,” the report states.

Solution in sight?
More research is needed to salvage CSA, according to the GAO recommendations.

The GAO recommends that the FMCSA conduct an analysis that identifies the limitations of the inspection and crash data used to calculate CSA scores. That analysis should include the variances in data collected on different motor carriers; the quality and quantity of the data; and the limitations in scores including their precision, confidence and reliability for enforcement purposes.

“Einstein said that the definition of insanity is doing the same thing over and over and expecting a different result,” Spencer said. “For the past 30 years, the federal officials pretty much have given the green light to a system of ‘turn them loose and try to catch them.’ Most new drivers are either untrained or significantly undertrained and asked to do things they are ill-equipped to do. It’s hardly a surprise that bad things happen.

“The approach coming out of Washington is basically nothing more than a coin flip to ensure responsibility and safety. Until the issues of driver training for new drivers and coercion in the industry are neutralized, any mathematical approach will fall short of achieving improved highway safety.”