Boundless Informant: the NSA's secret tool to track global surveillance data
http://www.guardian.co.uk/world/2013/jun/08/nsa-boundless-informant-global-dataminingGlenn Greenwald and Ewan McAskill
Revealed: The NSA's powerful tool for cataloguing data – including figures on US collection
The color scheme ranges from green (least subjected to surveillance) through yellow and orange to red (most surveillance). Note the '2007' date in the image relates to the document from which the interactive map derives its top secret classification, not to the map itself.
The National Security Agency has developed a powerful tool for recording and analysing where its intelligence comes from, raising questions about its repeated assurances to Congress that it cannot keep track of all the surveillance it performs on American communications.
The Guardian has acquired top-secret documents about the NSAdatamining tool, called Boundless Informant, that details and even maps by country the voluminous amount of information it collects from computer and telephone networks.
The focus of the internal NSA tool is on counting and categorizing the records of communications, known as metadata, rather than the content of an email or instant message.
The Boundless Informant documents show the agency collecting almost 3 billion pieces of intelligence from US computer networks over a 30-day period ending in March 2013. One document says it is designed to give NSA officials answers to questions like, "What type of coverage do we have on country X" in "near real-time by asking the SIGINT [signals intelligence] infrastructure."
An NSA factsheet about the program, acquired by the Guardian, says: "The tool allows users to select a country on a map and view the metadata volume and select details about the collections against that country."
Under the heading "Sample use cases", the factsheet also states the tool shows information including: "How many records (and what type) are collected against a particular country."
A snapshot of the Boundless Informant data, contained in a top secret NSA "global heat map" seen by the Guardian, shows that in March 2013 the agency collected 97bn pieces of intelligence from computer networks worldwide.
The heat map reveals how much data is being collected from around the world. Note the '2007' date in the image relates to the document from which the interactive map derives its top secret classification, not to the map itself.
Iran was the country where the largest amount of intelligence was gathered, with more than 14bn reports in that period, followed by 13.5bn from Pakistan. Jordan, one of America's closest Arab allies, came third with 12.7bn, Egypt fourth with 7.6bn and India fifth with 6.3bn.
The heatmap gives each nation a color code based on how extensively it is subjected to NSA surveillance. The color scheme ranges from green (least subjected to surveillance) through yellow and orange to red (most surveillance).
The disclosure of the internal Boundless Informant system comes amid a struggle between the NSA and its overseers in the Senate over whether it can track the intelligence it collects on American communications. The NSA's position is that it is not technologically feasible to do so.
At a hearing of the Senate intelligence committee In March this year, Democratic senator Ron Wyden asked James Clapper, the director of national intelligence: "Does the NSA collect any type of data at all on millions or hundreds of millions of Americans?"
"No sir," replied Clapper.
Judith Emmel, an NSA spokeswoman, told the Guardian in a response to the latest disclosures: "NSA has consistently reported – including to Congress – that we do not have the ability to determine with certainty the identity or location of all communicants within a given communication. That remains the case."
Other documents seen by the Guardian further demonstrate that the NSA does in fact break down its surveillance intercepts which could allow the agency to determine how many of them are from the US. The level of detail includes individual IP addresses.
IP address is not a perfect proxy for someone's physical location but it is rather close, said Chris Soghoian, the principal technologist with the Speech Privacy and Technology Project of the American Civil Liberties Union. "If you don't take steps to hide it, the IP address provided by your internet provider will certainly tell you what country, state and, typically, city you are in," Soghoian said.
That approximation has implications for the ongoing oversight battle between the intelligence agencies and Congress.
On Friday, in his first public response to the Guardian's disclosures this week on NSA surveillance, Barack Obama said that that congressional oversight was the American peoples' best guarantee that they were not being spied on.
"These are the folks you all vote for as your representatives in Congress and they are being fully briefed on these programs," he said. Obama also insisted that any surveillance was "very narrowly circumscribed".
Senators have expressed their frustration at the NSA's refusal to supply statistics. In a letter to NSA director General Keith Alexander in October last year, senator Wyden and his Democratic colleague on the Senate intelligence committee, Mark Udall, noted that "the intelligence community has stated repeatedly that it is not possible to provide even a rough estimate of how many American communications have been collected under the Fisa Amendments Act, and has even declined to estimate the scale of this collection."
At a congressional hearing in March last year, Alexander denied point-blank that the agency had the figures on how many Americans had their electronic communications collected or reviewed. Asked if he had the capability to get them, Alexander said: "No. No. We do not have the technical insights in the United States." He added that "nor do we do have the equipment in the United States to actually collect that kind of information".
Soon after, the NSA, through the inspector general of the overall US intelligence community, told the senators that making such a determination would jeopardize US intelligence operations – and might itself violate Americans' privacy.
"All that senator Udall and I are asking for is a ballpark estimate of how many Americans have been monitored under this law, and it is disappointing that the inspectors general cannot provide it," Wyden toldWired magazine at the time.
The documents show that the team responsible for Boundless Informant assured its bosses that the tool is on track for upgrades.
The team will "accept user requests for additional functionality or enhancements," according to the FAQ acquired by the Guardian. "Users are also allowed to vote on which functionality or enhancements are most important to them (as well as add comments). The BOUNDLESSINFORMANT team will periodically review all requests and triage according to level of effort (Easy, Medium, Hard) and mission impact (High, Medium, Low)."
Emmel, the NSA spokeswoman, told the Guardian: "Current technology simply does not permit us to positively identify all of the persons or locations associated with a given communication (for example, it may be possible to say with certainty that a communication traversed a particular path within the internet. It is harder to know the ultimate source or destination, or more particularly the identity of the person represented by the TO:, FROM: or CC: field of an e-mail address or the abstraction of an IP address).
"Thus, we apply rigorous training and technological advancements to combine both our automated and manual (human) processes to characterize communications – ensuring protection of the privacy rights of the American people. This is not just our judgment, but that of the relevant inspectors general, who have also reported this."
She added: "The continued publication of these allegations about highly classified issues, and other information taken out of context, makes it impossible to conduct a reasonable discussion on the merits of these programs."
Additional reporting: James Ball in New York and Spencer Ackerman in Washington
The National Security Agency: surveillance giant with eyes on America
The NSA is the best hidden of all the US intelligence services – and its secrecy has deepened as its reach has expanded
The very existence of National Security Agency (NSA) was not revealed for more than two decades after its establishment in 1952, and even now its structure and activities remain largely unknown. Hence its wry nickname: No Such Agency.
Of all the US intelligence services, it is has been the best hidden, and has prided itself on having the fewest leaks – at least until now. How many people does it employ? That is classified. Just how many people does it target? The NSA tells members of Congress that it does not have the tools to provide such figures.
When Harry Truman set up the NSA, it was exclusively aimed at monitoring communications abroad. The question that had exercised politicians and civil rights organisations since the Senate unveiled it in 1975 is to what extent its ferocious appetite for data has encompassed American citizens. General Lou Allen, the first NSA chief to appear in public, told Congress in the mid-1970s that the agency maintained lists of hundreds of names, including US citizens under surveillance for anti-war dissent or suspicious foreign connections.
As technology has evolved, so has the NSA's capacity to intercept an astonishing variety and volume of communications. Satellites scoop up calls and emails in the ether and beam the information back to earthbound receiving stations. One estimate suggests that each of these bases hoovers up roughly one billion emails, phone calls and other forms of correspondence every day, and the agency has up to 20 bases.
"This is not science fiction. It is happening now," a source with knowledge of the NSA said.
Domestic snooping exploded in scale after 9/11, when George W Bush authorised the agency to eavesdrop on Americans without the previous requirement for warrants. Within a few months of taking office in 2009, the Obama administration's Justice Department conceded that the agency had been guilty of "over-collection" of domestic communications but claimed the excess had been accidental.
With every passing administration, the NSA has ballooned. One well-informed estimate of its staffing levels is 100,000, of whom about 30,000 are military and the rest private contractors. Its headquarters is a vast edifice of smoked glass in Fort Meade, in the leafy Washington suburbs, with sizeable complexes in Georgia and Texas and overseas bases in Japan, Germany and the UK.
While the NSA is by far the biggest surveillance agency in the world, it shares some of its work with four other allies: Britain, Australia, New Zealand and Canada. Collectively, they are known as the "five eyes". Of the five, the biggest after the NSA is Britain's General Communications Headquarters (GCHQ).
The predecessor to the NSA was the short-lived Armed Forces Security Agency, which was set up in 1949. It was a relatively modest body compared with the mammoth the NSA has since become.
The young agency suffered some early embarrassments. In 1960, two of its staff defected to the Soviet Union. Three years later, a former NSA employee published code-breaking secrets in the Soviet paper Izvestia; the same year, an NSA employee killed himself while being investigated for selling secrets to Moscow.
Since then, internal security has been tightened significantly, but as the agency's secrecy deepened, its reach expanded relatively unchecked. It was a Democratic senator and lawyer, Frank Church, who in 1975 first raised the alarm at the agency's sprawling tentacles. During a series of hearings into the work of the intelligence agencies, he warned that the NSA's magnifying glass could be turned inwards on the American people.
"I don't want to see this country ever go across the bridge," he said. "I know the capacity that is there to make tyranny total in America, and we must see to it that this agency and all agencies that possess this technology operate within the law and under proper supervision, so that we never cross over that abyss. That is the abyss from which there is no return."
The Church Senate hearings led to the 1978 Foreign Intelligence Surveillance Act (Fisa), which required a warrant to conduct surveillance of communications within the US. A Fisa court, made up of a small group of judges appointed by the chief justice and located inside in the justice department, was given the job of deciding whether to grant warrants – it approves almost all requests.
In the years since 9/11, as the role of the NSA has snowballed, so has the debate over its operations. In 2005, the New York Times reported that the Bush administration had secretly authorised the NSA to eavesdrop on Americans and others inside the US, to search for terrorist activity without the Fisa court warrants. The order had been signed in 2002, but the newspaper cited about a dozen serving or former officials who expressed concern about the legality. The Times delayed publication of the article for a year because of government concerns about its impact.
In the face of the subsequent uproar, the Bush administration said it had ceased the warrantless surveillance in January 2007 and resumed the practice of requiring NSA warrants. In 2008, Congress loosened some of those constraints in the Fisa Amendment Act.
The massive surveillance programme has continued under the Obama administration, at home as well as abroad. And the culture of intense secrecy persists. For years, Senators Ron Wyden and Mark Udall have been demanding to know just how many people inside the US have been spied on by the NSA. No answer has been forthcoming
NSA Prism program taps in to user data of Apple, Google and others
• Top-secret Prism program claims direct access to servers of firms including Google, Apple and Facebook
• Companies deny any knowledge of program in operation since 2007
• Companies deny any knowledge of program in operation since 2007
The National Security Agency has obtained direct access to the systems of Google, Facebook, Apple and other US internet giants, according to a top secret document obtained by the Guardian.
The NSA access is part of a previously undisclosed program calledPrism, which allows officials to collect material including search history, the content of emails, file transfers and live chats, the document says.
The Guardian has verified the authenticity of the document, a 41-slide PowerPoint presentation – classified as top secret with no distribution to foreign allies – which was apparently used to train intelligence operatives on the capabilities of the program. The document claims "collection directly from the servers" of major US service providers.
Although the presentation claims the program is run with the assistance of the companies, all those who responded to a Guardian request for comment on Thursday denied knowledge of any such program.
In a statement, Google said: "Google cares deeply about the security of our users' data. We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government 'back door' into our systems, but Google does not have a back door for the government to access private user data."
Several senior tech executives insisted that they had no knowledge of Prism or of any similar scheme. They said they would never have been involved in such a program. "If they are doing this, they are doing it without our knowledge," one said.
An Apple spokesman said it had "never heard" of Prism.
The NSA access was enabled by changes to US surveillance law introduced under President Bush and renewed under Obama in December 2012.
The program facilitates extensive, in-depth surveillance on live communications and stored information. The law allows for the targeting of any customers of participating firms who live outside the US, or those Americans whose communications include people outside the US.
It also opens the possibility of communications made entirely within the US being collected without warrants.
Disclosure of the Prism program follows a leak to the Guardian on Wednesday of a top-secret court order compelling telecoms provider Verizon to turn over the telephone records of millions of US customers.
The participation of the internet companies in Prism will add to the debate, ignited by the Verizon revelation, about the scale of surveillance by the intelligence services. Unlike the collection of those call records, this surveillance can include the content of communications and not just the metadata.
Some of the world's largest internet brands are claimed to be part of the information-sharing program since its introduction in 2007. Microsoft – which is currently running an advertising campaign with the slogan "Your privacy is our priority" – was the first, with collection beginning in December 2007.
It was followed by Yahoo in 2008; Google, Facebook and PalTalk in 2009; YouTube in 2010; Skype and AOL in 2011; and finally Apple, which joined the program in 2012. The program is continuing to expand, with other providers due to come online.
Collectively, the companies cover the vast majority of online email, search, video and communications networks.
The extent and nature of the data collected from each company varies.
Companies are legally obliged to comply with requests for users' communications under US law, but the Prism program allows the intelligence services direct access to the companies' servers. The NSA document notes the operations have "assistance of communications providers in the US".
The revelation also supports concerns raised by several US senators during the renewal of the Fisa Amendments Act in December 2012, who warned about the scale of surveillance the law might enable, and shortcomings in the safeguards it introduces.
When the FAA was first enacted, defenders of the statute argued that a significant check on abuse would be the NSA's inability to obtain electronic communications without the consent of the telecom and internet companies that control the data. But the Prism program renders that consent unnecessary, as it allows the agency to directly and unilaterally seize the communications off the companies' servers.
A chart prepared by the NSA, contained within the top-secret document obtained by the Guardian, underscores the breadth of the data it is able to obtain: email, video and voice chat, videos, photos, voice-over-IP (Skype, for example) chats, file transfers, social networking details, and more.
The document is recent, dating to April 2013. Such a leak is extremely rare in the history of the NSA, which prides itself on maintaining a high level of secrecy.
The Prism program allows the NSA, the world's largest surveillance organisation, to obtain targeted communications without having to request them from the service providers and without having to obtain individual court orders.
With this program, the NSA is able to reach directly into the servers of the participating companies and obtain both stored communications as well as perform real-time collection on targeted users.
The presentation claims Prism was introduced to overcome what the NSA regarded as shortcomings of Fisa warrants in tracking suspected foreign terrorists. It noted that the US has a "home-field advantage" due to housing much of the internet's architecture. But the presentation claimed "Fisa constraints restricted our home-field advantage" because Fisa required individual warrants and confirmations that both the sender and receiver of a communication were outside the US.
"Fisa was broken because it provided privacy protections to people who were not entitled to them," the presentation claimed. "It took a Fisa court order to collect on foreigners overseas who were communicating with other foreigners overseas simply because the government was collecting off a wire in the United States. There were too many email accounts to be practical to seek Fisas for all."
The new measures introduced in the FAA redefines "electronic surveillance" to exclude anyone "reasonably believed" to be outside the USA – a technical change which reduces the bar to initiating surveillance.
The act also gives the director of national intelligence and the attorney general power to permit obtaining intelligence information, and indemnifies internet companies against any actions arising as a result of co-operating with authorities' requests.
In short, where previously the NSA needed individual authorisations, and confirmation that all parties were outside the USA, they now need only reasonable suspicion that one of the parties was outside the country at the time of the records were collected by the NSA.
The document also shows the FBI acts as an intermediary between other agencies and the tech companies, and stresses its reliance on the participation of US internet firms, claiming "access is 100% dependent on ISP provisioning".
In the document, the NSA hails the Prism program as "one of the most valuable, unique and productive accesses for NSA".
It boasts of what it calls "strong growth" in its use of the Prism program to obtain communications. The document highlights the number of obtained communications increased in 2012 by 248% for Skype – leading the notes to remark there was "exponential growth in Skype reporting; looks like the word is getting out about our capability against Skype". There was also a 131% increase in requests for Facebook data, and 63% for Google.
The NSA document indicates that it is planning to add Dropbox as a PRISM provider. The agency also seeks, in its words, to "expand collection services from existing providers".
The revelations echo fears raised on the Senate floor last year during the expedited debate on the renewal of the FAA powers which underpin the PRISM program, which occurred just days before the act expired.
Senator Christopher Coons of Delaware specifically warned that the secrecy surrounding the various surveillance programs meant there was no way to know if safeguards within the act were working.
"The problem is: we here in the Senate and the citizens we represent don't know how well any of these safeguards actually work," he said.
"The law doesn't forbid purely domestic information from being collected. We know that at least one Fisa court has ruled that the surveillance program violated the law. Why? Those who know can't say and average Americans can't know."
Other senators also raised concerns. Senator Ron Wyden of Oregon attempted, without success, to find out any information on how many phone calls or emails had been intercepted under the program.
When the law was enacted, defenders of the FAA argued that a significant check on abuse would be the NSA's inability to obtain electronic communications without the consent of the telecom and internet companies that control the data. But the Prism program renders that consent unnecessary, as it allows the agency to directly and unilaterally seize the communications off the companies' servers.
When the NSA reviews a communication it believes merits further investigation, it issues what it calls a "report". According to the NSA, "over 2,000 Prism-based reports" are now issued every month. There were 24,005 in 2012, a 27% increase on the previous year.
In total, more than 77,000 intelligence reports have cited the PRISM program.
Jameel Jaffer, director of the ACLU's Center for Democracy, that it was astonishing the NSA would even ask technology companies to grant direct access to user data.
"It's shocking enough just that the NSA is asking companies to do this," he said. "The NSA is part of the military. The military has been granted unprecedented access to civilian communications.
"This is unprecedented militarisation of domestic communications infrastructure. That's profoundly troubling to anyone who is concerned about that separation."
A senior administration official said in a statement: "The Guardian and Washington Post articles refer to collection of communications pursuant to Section 702 of the Foreign Intelligence Surveillance Act. This law does not allow the targeting of any US citizen or of any person located within the United States.
"The program is subject to oversight by the Foreign Intelligence Surveillance Court, the Executive Branch, and Congress. It involves extensive procedures, specifically approved by the court, to ensure that only non-US persons outside the US are targeted, and that minimize the acquisition, retention and dissemination of incidentally acquired information about US persons.
"This program was recently reauthorized by Congress after extensive hearings and debate.
"Information collected under this program is among the most important and valuable intelligence information we collect, and is used to protect our nation from a wide variety of threats.
"The Government may only use Section 702 to acquire foreign intelligence information, which is specifically, and narrowly, defined in the Foreign Intelligence Surveillance Act. This requirement applies across the board, regardless of the nationality of the target."
Additional reporting by James Ball and Dominic Rushe
Facebook and Google insist they did not know of Prism surveillance program
Larry Page and Mark Zuckerberg sharply deny knowledge of Prism until Thursday even as Obama confirms program's existence
America's tech giants continued to deny any knowledge of a giant government surveillance programme called Prism, even as presidentBarack Obama confirmed the scheme's existence Friday.
With their credibility about privacy issues in sharp focus, all the technology companies said to be involved in the program issued remarkably similar statements.
All said they did not allow the government "direct access" to their systems, all said they had never heard of the Prism program, and all called for greater transparency.
In a blogpost titled 'What the…?' Google co-founder Larry Page and chief legal officer David Drummond said the "level of secrecy" around US surveillance procedures was undermining "freedoms we all cherish."
"First, we have not joined any program that would give the US government – or any other government – direct access to our servers. Indeed, the US government does not have direct access or a 'back door' to the information stored in our data centers. We had not heard of a program called Prism until yesterday," they wrote.
"Second, we provide user data to governments only in accordance with the law. Our legal team reviews each and every request, and frequently pushes back when requests are overly broad or don't follow the correct process."
The Google executives said they were also "very surprised" to learn of the government order made to obtain data from Verizon, first disclosed by the Guardian. "Any suggestion that Google is disclosing information about our users' internet activity on such a scale is completely false," they wrote.
Mark Zuckerberg, the founder and CEO of Facebook, described the press reports about Prism as "outrageous". He insisted that the Facebook was not part of any program to give the US government direct access to its servers.
He said: "Facebook is not and has never been part of any program to give the US or any other government direct access to our servers. We have never received a blanket request or court order from any government agency asking for information or metadata in bulk, like the one Verizon reportedly received. And if we did, we would fight it aggressively. We hadn't even heard of Prism before yesterday."
Zuckerberg also called for greater transparency. "We strongly encourage all governments to be much more transparent about all programs aimed at keeping the public safe. It's the only way to protect everyone's civil liberties and create the safe and free society we all want over the long term."
Yahoo said: "We do not provide the government with direct access to our servers, systems, or network."
The leaked National Security Agency (NSA) document obtained by the Guardian claims Prism operates with the "assistance of communications providers in the US".
The document names AOL, Apple, Facebook, Google, Microsoft, PalTalk and Yahoo and gives dates when they "joined" the scheme, aimed at intercepting data from people outside the US.'' The presentation talks of "legally compelled collection" of data.
All the companies involved have now denied knowledge of the scheme to the Guardian.
In one slide, the presentation identifies two types of data collection: Upstream and Prism. Upstream involves the collection of communications on "fibre cables and infrastructure as data flows past." Prism involves: "Collection directly from the servers of these US service providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple."
Obama confirmed the existence of the scheme Friday and said Congress was "fully apprised" of the situation and that it was being conducted legally with a "whole range of safeguards involved".
But despite Obama's acknowledgment, senior figures said they remained puzzled and surprised by the news. Speaking off the record one said their company regularly complied with subpoenas for information but had never allowed "collection directly" from their servers.
Some speculated that the wording of the document was incorrect or that the author had over-hyped the scheme.
Security experts and civil liberty figures were less convinced. "I was assuming that these tech companies were just lying," said security guru Bruce Schneier. "That's the most obvious explanation."
"Could it possibly be that there's a department within these companies that hides this from the executives? Maybe," he said. "I don't know, we don't know. This points to the problem here. There's so much freaking secrecy that we don't know enough to even know what is going on."
He said he was not surprised by the news. "There are no surprises here. We all knew what was going on and now they have finally admitted it."
"The NSA would not have done this surreptitiously, they want the tech companies on their side," said Jameel Jaffer, director of the American Civil Liberties Union (ACLU). "I can't make sense of their statements at all."
He said it was clear that tech companies in general were more than happy to co-operate with the US authorities and said he was puzzled why there seemed to be such a gap between the two sides' story.
Ali Reza Manouchehri, CEO and co-founder of MetroStar Systems, an IT consultant that works closely with government agencies, said: "There are situations that come up where they have to communicate with the security agencies. At the end of the day they are working in the interest of national security."
"I can't comment on what's going on inside the company. It's hard for me to believe that Google doesn't know," he said. "It is either transparent or it is surreptitious. It is hard for me to believe that at this level, at this volume it is surreptitious." He said if the companies really did not know then "we have some serious issues."
The news has sparked widespread concern in the US. Nearly 20,000 people have signed a petition at Progressive Change Campaign Committee calling on Congress to hold investigations.
No comments:
Post a Comment