How 2.6M COVID tests on DPH site revealed a nearly 2-year data issue

DPH says daily reports of tests and cases on its website are accurate

Credit: Ben Gray

Credit: Ben Gray

Editor’s note: Story updated to note continued data publishing issues that extended into Thursday.

For nearly two years, the Georgia Department of Public Health (DPH) has posted daily coronavirus test data on its public website.

On the heels of a holiday weekend predicted to produce huge numbers of COVID-19 infections, DPH reported some eye-popping figures Tuesday: more than 2.6 million tests and nearly 400,000 positives since its last update four days earlier on New Year’s Eve.

At first glance, the numbers – which would amount to nearly 20% of all tests administered since the pandemic started – looked like a blunder. And they didn’t match daily test or infection data posted elsewhere on the website.

A DPH press release issued Wednesday night, more than 24 hours later, labeled it a “coding error.”

But The Atlanta Journal-Constitution has learned a data processing issue has caused DPH to under-report daily COVID-19 tests and positive results in at least one prominent location on its website for nearly two years.

And on Tuesday, a coding change caused DPH’s website to catch up in one gigantic swoop.

DPH spokeswoman Nancy Nydam said Wednesday the problem does not reflect an overall undercount of tests administered or infections — numbers relied upon by schools, local officials and businesses to make decisions such as when to open or close.

Though the prominent column of daily testing information was underrepresenting data, Nydam said the testing and case information was accounted for accurately in all other areas of the dashboard, meaning it was not lost.

Since the beginning of the pandemic, numerous independent experts have questioned Georgia’s accounting of coronavirus cases, tests, deaths and vaccinations. Some of the state’s handling of data has been held up for ridicule.

The data mishaps present real-world consequences, independent health care experts say.

DPH delayed reporting of new COVID-19 data Wednesday for more than five hours because of the daily test counting issue.

On Thursday, hours after the AJC published an initial version of this story online, DPH announced it would not update the portal for the day. Instead, the agency reported limited data, including more than 23,000 new infections and 51 confirmed COVID-19 deaths in an afternoon press release.

DPH also delayed its Monday report, which included the holiday weekend, because of a separate matter involving an “overwhelming” volume of new data amid a surge in the virus that’s led to record high numbers of infections.

Amber Schmidtke, a public health researcher who tracks Georgia’s coronavirus epidemic in her widely read newsletter, said DPH has been underfunded for years and she urged the Legislature to spend more to improve the agency’s resources.

“I’d like to give the benefit of the doubt that they’re doing their best with the resources they have,” she said of DPH. “Even two years into a pandemic, public health is not a priority to the policymakers who fund it.”

DPH did not make Dr. Kathleen Toomey, the state’s commissioner of public health, available for an interview.

DPH data troubles

The AJC has chronicled a number of issues at DPH in how it tracks and visualizes pandemic data.

  • In April 2020, the AJC found data on infections and deaths in nursing homes was inaccurate or severely outdated. The state also had difficulty determining race and ethnicity data of infected individuals at a time when COVID-19 was hitting minorities particularly hard. Independent public health experts said detailed information is vital to understanding trends.
  • In April 2020, the AJC found the DPH did not have race data from drive-up testing sites because it did not realize it had left the race field off of its own form.
  • In May 2020, the AJC reported Georgia combined data about diagnostic and antibody tests, making it appear the state was testing more people for active infections than it was.
  • In May 2020, a chart suggested infections in the biggest counties were dropping over time when no clear downward trend was evident because the dates were out of order.
  • In July 2020, the AJC found DPH only knew the race and ethnicity in fewer than 7 out of 10 confirmed cases of COVID-19.
  • In November 2020, an AJC analysis found that Georgia virus rates were likely a third higher than the official tally. As many Georgians were relying on antigen tests, DPH’s dashboard at the time did not include daily results for those tests in its statewide or county-level charts and maps of positive cases. DPH ultimately added this information to its dashboard in the months after the AJC story.
  • In late March and early April 2021, DPH took down its vaccine dashboard for nearly two weeks citing ongoing system and data transfer issues.
  • In April 2021, the AJC found the state consistently reported higher vaccination rates, particularly among seniors, because DPH used an older and smaller population estimate than the CDC.

DPH’s dashboard charts numerous pandemic data points including tests, infections and deaths for the state and all 159 counties.

DPH’s tallies of cumulative PCR tests, positives and percentage of positive tests are accurate, Nydam said Wednesday. DPH said the whopping testing figures reported Tuesday have been accounted for in cumulative totals all along.

The dashboard’s charts showing testing data statewide and by county by day also are accurate, Nydam said.

But on Thursday, AJC confirmed a reader tip of another apparent 2 million-plus discrepancy between two other DPH data sets. One tracks tests since the start of the pandemic by the date DPH receives the report and the other by date of sample collection.

DPH did not respond to questions about this difference before press time and it is unclear what the source of the discrepancy is.

Nydam said on Wednesday DPH’s data issue did not affect DPH data on cases, deaths and other key metrics.

In that case, researchers and media organizations that pull raw data from other parts of DPH’s site also likely were not affected by the data issue.

The data processing issue affects only one part of the dashboard – a column in its COVID-19 testing chart titled “PCR/Molecular Reported Today,” Nydam said.

Credit: J. Scott Trubey

Credit: J. Scott Trubey

Before Tuesday, that column failed to capture the 2.6 million tests, meaning it is likely for nearly two years the daily total tests reported in that column were off by several thousand each day. Positive tests would likely have been off from actual reports by hundreds with each daily update.

On Wednesday, DPH delayed publication of its daily status update and Nydam said DPH would temporarily remove the daily PCR test result column until the matter can be resolved. All other data, she said, would continue to be reported as usual.

A day later, DPH scuttled its Thursday update.

“DPH is actively working to resolve the issue with the COVID-19 testing numbers received through (Electronic Laboratory Reporting) to ensure we are providing accurate and transparent data about COVID-19 in Georgia,” Nydam said in a news release Thursday. “We ask for your continued patience as we address this issue.”

Daily percent positive information through Wednesday can still be found on the DPH website in its “COVID-19 Over Time” graph, the agency said.

In its news releases, DPH did not acknowledge the “PCR/Molecular Reported Today” column included incomplete data for nearly two years.

Independent public health experts say Georgia’s failure to invest in DPH and its broader health care delivery and surveillance network has hampered response to a virus that has resulted in more than 26,000 confirmed COVID-19 deaths and more than 5,000 more deaths in which coronavirus is suspected.

State Rep. Rebecca Mitchell, D-Snellville, who is also an epidemiologist accustomed to handling large data sets, raised questions about the 2.6 million tests reported Tuesday when the state’s cumulative test figures did not increase by a comparable amount.

Asked if the state’s data miscues are a result inadequate resources, Mitchell said, “100 percent.”

DPH scientists should have caught such a problem with more resources, she said.

“We do understand they’re overstressed, and perfection isn’t an attainable goal,” Mitchell said.

But, she said, Georgians expect transparency.

Each day, providers and labs report testing data to DPH and that data goes into its charts and cumulative totals. But the tally of tests reported each day involves a more complicated computer query, Nydam said.

DPH runs a data query that takes about 15 minutes to determine how many tests arrived since the last report. Those tests are given a special timestamp for that query. But tests that come in during that 15-minute processing period do not, falling between the cracks in the daily report tabulation.

The tests not given the special timestamp were still counted in the state’s cumulative totals and those data also appear accurately elsewhere on the dashboard, Nydam said.

Nydam said an update in the state’s systems on New Year’s Eve bogged down a query for daily test results, taking hours for what normally took minutes.

DPH officials simplified the query, resulting in the appearance Tuesday of the 2.6 million tests dating as far back as the start of the pandemic that lacked the timestamp.

DPH only addressed the daily coronavirus test issue after the AJC started asking questions.