IT Problems ‘May Have Negatively Affected Data Quality’ of 2010 Census, Says Inspector General

By Edwin Mora | February 1, 2011 | 6:33 PM EST

A copy of a 2010 Census form. (AP Photo/Idaho Press-Tribune, Charlie Litchfield)

( - Problems with a Census Bureau computer system known as the Paper-Based Operations Control System (PBOCS) “may have negatively affected data quality” in the 2010 Census, according to a report from the inspector general of the Commerce Department.

The PBOCS is the computer program the Census Bureau used to manage the flow of paper questionnaires and other documents that part-time Census enumerators used when carrying out field operations to count people who had failed to return questionnaires to the Census Bureau via mail—what the Census Bureau calls its Non-Response Follow-up (NRFU) operation.

One of the lead findings in the new inspector general’s report on the Census is: “IT Systems Instability Caused Higher Costs and May Have Negatively Affected Data Quality.”

“IT systems instability caused higher costs and may have negatively affected data quality,” says the report. “Our February and May quarterly reports raised questions about the viability of PBOCS, Census’s primary tool for field workload planning and reporting. Our field observations confirmed the expected: increased cost and potential data errors because of PBOCS’ inadequacies.”

“The most frequent complaint concerned checking in questionnaires that enumerators had returned to the local offices,” said the inspector general’s report. “Questionnaires are checked in to ensure that all questionnaires sent out into the field are returned, and that the appropriate data had been entered on each questionnaire before shipping them to the data-capture centers. Office clerks had difficulty checking in questionnaires primarily because PBOCS’ availability was limited due to unacceptably slow performance or complete outages. Total PBOCS outages during NRFU amounted to nearly 80 hours, mostly in May, the first month of the operation. Outages and slow performance caused a large backlog of over 10 million questionnaires awaiting check-in, which delayed the NRFU-Re-interview operation.”

Despite the inspector general's findings, the Census Bureau maintains that the IT problems did not negatively affect the quality of the data collected during its Non-Response Follow-Up operation.

The NRFU, which lasted from the beginning of May through the end of June, involved temporary federal workers seeking to conduct door-to-door interviews at 47 million households that did not return Census form by mail.

Data collected by the Census is used to determine the number of U.S. House seats allocated to each state and the distribution of more than $400 billion in federal funds to local communities.

The NRFU temporary workers used paper forms to collect data from households in the field. The flow of these forms in and out of regional Census Bureau offices and to data compilation centers was supposed to be managed by the PBOCS computer program—which did not work properly.

“Census was forced to work in what can best be described as crisis management mode, repairing technical problems and developing clerical workarounds and automated contingencies in real time to compensate for the loss of PBOCS’ functionality,” said the inspector general's report.

“As a result of this substantial effort, Census was able to complete NRFU on time, but not without increased costs and potential data quality issues,” said the IG report.

Earlier, even before the NRFU operation began, the inspector general had warned that problems with the Census Bureau’s computer program placed the accuracy of the NRFU count at risk.

“If PBOCS is not ready or if additional actions are not taken, the operation could be adversely affected, resulting in increased costs and reduced accuracy of the population count,” the inspector general warned in a February 2010 report to Congress.

“Unless PBOCS stability improves substantially, the cost of the NRFU operation, its timely completion, and the accuracy of its count are at risk,” the inspector general warned again in a May 2010 report to Congress.

After the NRFU operation concluded in the late summer of 2010, asked Census Director Robert Groves if the accuracy of the Census was affected by the IT problems cited in the inspector general’s reports.

At that time, Groves told there was “no evidence” to suggest that the IT problems had affected the quality of the data collected by the Census.

The Census Bureau maintains that position today despite the new inspector general’s statement that “field observations confirmed the expected: increased cost and potential data errors because of PBOCS’ inadequacies.”

Reacting to the inspector general’s report, Census Bureau Spokesman Michael Cook told, “We instituted improved quality assurance methods, and carried out numerous quality checks in the 2010 Census, and we have no reason to believe that IT problems led to accuracy errors.”

The inspector general’s report, by contrasts, specifically says that the IT problems impaired the Census Bureau’s quality control operations.

“As the backlog of questionnaires grew, cases could not be selected for re-interview, increasing the time lag between the original and follow-up interviews,” said the inspector general. “This time lag can adversely affect the quality of the data collected in the follow-up interviews because the more time that elapses, the less reliable the respondent’s memory of what was initially reported. And the slow pace of identifying addresses to re-interview delayed the bureau from retraining or removing enumerators performing questionable work.”