Through Freedom of Information Act requests and interviews with officials at numerous agencies, The Wall Street Journal has reconstructed the clash over the counterterrorism program within the administration of President Barack Obama. The debate was a confrontation between some who viewed it as a matter of efficiency—how long to keep data, for instance, or where it should be stored—and others who saw it as granting authority for unprecedented government surveillance of U.S. citizens.

The rules now allow the little-known National Counterterrorism Center to examine the government files of U.S. citizens for possible criminal behavior, even if there is no reason to suspect them. That is a departure from past practice, which barred the agency from storing information about ordinary Americans unless a person was a terror suspect or related to an investigation.

Now, NCTC can copy entire government databases—flight records, casino-employee lists, the names of Americans hosting foreign-exchange students and many others. The agency has new authority to keep data about innocent U.S. citizens for up to five years, and to analyze it for suspicious patterns of behavior. Previously, both were prohibited. Data about Americans “reasonably believed to constitute terrorism information” may be permanently retained.

The changes also allow databases of U.S. civilian information to be given to foreign governments for analysis of their own. In effect, U.S. and foreign governments would be using the information to look for clues that people might commit future crimes.