Wednesday, February 25, 2009

Website Maps Global Web Blocks

Herdict uses volunteers to monitor Internet censorship, filtering, and outages.

By David Talbot

Internet censorship occurs worldwide for a broad range of reasons, but the exact dimensions of the problem can be hard to document. A website launching today aims to tap the power of "crowd sourcing"--fielding and aggregating reports from volunteers--to provide real-time data on the state of Net filtering.

"We hope to have an ongoing and increasingly refined map of blockages around the world, as they happen," says Jonathan Zittrain, a Harvard law professor and cofounder of the Berkman Center for Internet and Society. "It might not just be blocks occasioned by a government trying to get rid of content," he adds. "It might be a company filtering stuff from its employees, and it might also be takedowns of content"--for example, videos that are alleged to infringe on copyright.

The site extends the work of the OpenNet Initiative, a joint project coordinated by institutes at Harvard, the University of Toronto, the University of Oxford, and the University of Cambridge to document Web filtering around the world. To date, the efforts of this project have been labor intensive: researchers typically dispatched volunteers within certain countries to check the accessibility of websites containing sensitive information. This method contributed to a study called Access Denied that produced a map documenting evidence of filtering within about 40 nations three years ago.

Such efforts will continue, but they will now be augmented by citizen reports, contributed via a site called Herdict (the name is a play on "the verdict of the herd," Zittrain says). The site aggregates reports and displays trends over time, showing which sites are blocked in which countries. Herdict is only designed to show whether or not a site is accessible; it cannot say whether the problem is due to government censorship, an Internet service provider's filtering policy, or nothing more than a severed cable. However, the researchers argue that such answers can be inferred by monitoring broader trends. "We are operating completely in the dark right now," says Zittrain. "But it's something that is hidden in plain sight, because any individual knows his or her own experience."

In theory, data collected by the site could also be used to compare the service quality of different ISPs, he adds.

The effort does present a potential catch-22, however: what happens if the site itself is blocked? Zittrain notes that reports could still reach Herdict via other means, such as Twitter. He adds, "When we get blocked, it will be our first milestone of success: we've done enough that we are worth blocking!"