Systematic Failures of Chemical Safety Research at Environmental Protection Agency By Angela Logomasini, Ph.D.
“If at first you don’t succeed, try, try again” could be the motto of one of the key research programs at the U.S. Environmental Protection Agency known as the Integrated Risk Information System (IRIS). EPA staff have been working for nearly a decade to improve the scientific process of this program, but have yet to get it right.
IRIS has proven controversial because of its history of relatively low productivity and sloppy research. Despite reform efforts during the past decade, the Government Accountability Office (GAO) continues to list IRIS on its “High Risk List” of programs that are vulnerable to “fraud, waste, abuse and mismanagement.”
EPA officials created IRIS administratively in 1984 to provide information on chemical risks to aid other EPA programs that needed such data for regulatory purposes. IRIS first produced a database of chemicals and relevant data. The office eventually went on to conduct risk assessments for various chemicals, but there are no legal guidelines for such assessments. In 2000, the Office of Management and Budget began to provide some oversight of the program. The GAO raised concerns about IRIS’s productivity and its procedures more than a decade ago, and since then IRIS reform has remained a hot topic in Washington, subject of more GAO reports, an Inspector General report, and congressional hearings.
A 2011 review of the IRIS risk assessment for formaldehyde turned up the heat for reform as a National Academies of Sciences review panel and report pointed out serious methodological failures and IRIS’s “recurring problems with clarity and transparency of the methods.” The NAS committee made a number of recommendations for improving the IRIS program, including the recommendation that it apply literature review procedures referred to as “systematic review.”
Originally designed for review of medical research, systematic reviews go beyond simple narrative literature reviews, employing procedures to ensure a more a comprehensive review of the literature in an attempt to reduce researcher bias involved with selecting studies. NAS also urged IRIS officials to operate in a more transparent way, among other recommendations.
Of course, operating in a systematic and transparent way should be a no brainer for anyone interested in producing solid and useful science, and EPA should have changed its procedures quickly. Yet three years after the first NAS review, EPA was still trying to make its research systematic when the NAS conducted yet another review, indicating that the agency made some improvements but still had work to do.
In January 2017, as IRIS staff were still trying to figure out how to be systematic and transparent, the Obama EPA selected new leadership for the program, placing former Environmental Working Group senior scientist Kristina Thayer at the helm. Shortly before leaving office, the Obama EPA also selected Tina Bahadori to head the National Center for Environmental Assessment (NCEA), which houses IRIS within EPA’s Office of Research and Development. Supposedly, Thayer and Bahadori picked up speed with improvements.
Yet concern about IRIS’s slow progress and past excesses advanced efforts to cut its budget and maybe even eliminate it all together. Earlier this year, President Trump’s budget proposed significant budget cuts and the Senate Omnibus reconciliation bill proposed IRIS elimination.
IRIS elimination is no longer only driven by the program’s inadequacies; IRIS has become much less relevant now that Congress reauthorized and expanded the Toxic Substances Control Act in 2016. This reform vastly expanded the EPA’s power to collect data from industry and use it for chemical safety assessments, and EPA has already begun setting priorities for this program. Those efforts will be extensive and governed by congressionally mandated, rigorous scientific standards, which is something IRIS lacks.
Seeing the writing on the wall, Thayer and Bahadori apparently hatched a plan to save IRIS by calling for yet another NAS review. For a day and a half last February, Thayer and Bahadori briefed the NAS panel on all the procedural reforms and activities they were implementing at EPA. NAS released this review on April 11, and this time EPA got what it had been trying to achieve for many years: praise for its reforms. The NAS review allegedly shows that the IRIS has finally made some real progress, but a closer look suggests that the NAS report is little more than a rubber stamp.
EPA achieved this end by keeping the scope of the NAS review extremely narrow, just calling on the committee to assess whether the “current trajectory of the program agrees with past recommendations of the National Academies” in 2011 and 2014. Note the review only assessed if EPA was on the right path, not whether it had reached its goals. The NAS report explains further that the committee “was not asked to evaluate the overall value of the IRIS program,” and it “was not tasked with conducting a comprehensive review of the IRIS program.” In other words, all the NAS was asked to do was validate the efforts EPA has undertaken to improve the process, not whether it finally achieved its goals or whether it had substantially improved its assessments.