Government Information Invisible to Popular Search Engines, says Report

Vital government information appears "invisible" to millions of Americans who are combing the Internet and looking for answers via the most popular search engines, according to a report released today by the Center for Democracy and Technology (CDT) and OMB Watch.

The report, Hiding in Plain Sight: Why Important Government Information Cannot Be Found through Commercial Search Engines, highlights the shortcomings some federal agencies exhibit when trying to comply with the mandates of the E-Government Act of 2002, a landmark law that promotes access to government information and services.

"Unquestionably, the E-Government Act has changed the way that the public interacts with the government," said Ari Schwartz, CDT's deputy director. "Unfortunately, despite the availability of an easy technological fix, many key governmental information sources remain 'hidden in plain sight,' from the very search engines that the public is most likely to use."

While the report points to critical gaps in online access to government information, it makes no judgment call as to the reason this information is inaccessible, other than to expose a simple technological roadblock as the culprit; an equally simple technological fix is also noted in the report.

"It is unclear if agencies know there is a roadblock between the public and their information and have not taken the adequate steps to correct the problem, or if the agencies simply do not realize that their important information is not being found and indexed by search engines," said Sean Moulton, director of federal information policy for OMB Watch. "In today's Internet age, either answer is unacceptable."

The report uses several search examples that Americans might expect to result in access to trustworthy government information. Instead, the results overlook a vast amount of useful government information. Among those results:

• A search for "New York radiation" does not find basic FEMA and DHS information about current conditions and monitoring.
• A search to help grandparents with a question about their rights to visit their grandchildren does not turn up an article specifically answering that question that is located on the Web site of the Administration for Children & Families.
• A search for "small farm loans" turns up commercial offers for loans and statistics about government loans, but not most of the major federal government programs designed to help fund small farms.
  

Based on the findings of the report, CDT and OMB Watch have several recommendations for the federal government. Each of these would encourage greater accessibility of government information by making it more searchable:

• Congress should pass the E-Government reauthorization act, which would require the Office of Management and Budget (OMB) to create best practices to encourage searchability of federal Web sites.
• OMB should officially recognize the importance of commercial search engines to Internet users and work with the CIO Council to adopt policies to help users find information.
• Agencies should adopt an information policy that makes public accessibility of online content and resources a priority.
• Agencies should create sitemaps of content on their sites, with special attention given to materials stored in databases and accessible only through drop-down menus. For example, many agencies have FAQ databases that are not accessible to search crawlers but contain very succinct and useful answers to common questions.
• Agencies should review their use of robots.txt files in order to ensure they are used in the least restrictive way possible. Every effort should be made to include, rather than exclude, materials from the Web site, whether materials were excluded purposefully or accidentally in the past.
  

"We urge the Committee to work with us to pressure agencies that have not made public information available through search engines to do so immediately," Schwartz said.