Taking Spying to Higher Level, Agencies Look for More Ways to Mine Data

A small group of National Security Agency officials slipped into Silicon Valley on one of the agency's periodic technology shopping expeditions this month.
On the wish list, according to several venture capitalists who met with the officials, were an array of technologies that underlie the fierce debate over the Bush administration's anti-terrorist eavesdropping program: computerized systems that reveal connections between seemingly innocuous and unrelated pieces of information.
The tools they were looking for are new, but their application would fall under the well-established practice of data mining: using mathematical and statistical techniques to scan for hidden relationships in streams of digital data or large databases.
Supercomputer companies looking for commercial markets have used the practice for decades. Now intelligence agencies, hardly newcomers to data mining, are using new technologies to take the practice to another level.
But by fundamentally changing the nature of surveillance, high-tech data mining raises privacy concerns that are only beginning to be debated widely. That is because to find illicit activities it is necessary to turn loose software sentinels to examine all digital behavior whether it is innocent or not.