For the first time in its history, the United Kingdom has launched its first-ever military-style cyber campaign against an adversary, according to the director of the country’s primary cyber security agency. The target of the campaign was the Islamic State, the militant Sunni Muslim group that is also known as the Islamic State of Iraq and Syria (ISIS). The existence of the all-out cyber war was announced last week by Jeremy Fleming
Privacy International, a non-government organisation (NGO) and campaigning group, is expected to argue in the UK’s most secret court today that contractors with privileged access to intelligence service computer systems pose a clear risk to sensitive data gathered by GCHQ and the intelligence services.
For example, Edward Snowden used his systems administrator rights as external contractor to the US National Security Agency (NSA) to download “Top Secret Strap” documents from GCHQ.
In another case, a contractor working for the NSA reportedly leaked hacking tools to the Russian antivirus software company Kaspersky Lab. The contractor claimed to have taken NSA software home to work on, on his personal computer. Kaspersky’s software identified malware attributed to the “Equation Group”, the code name for the security agency’s hacking team.
I get it now.
The intelligence community’s growing reliance on contractors
GCHQ, MI5 and MI6 have become increasingly reliant on external contractors over the past decade. Between 2011 and 2016 their combined spending on consultants and contractors grew from 20% of the overall intelligence agency budget to 30%.
The Cheltenham-based agency is expanding rapidly and, according to the latest figures available, in 2016/16 was spending £70m a year on contractors to fill staff vacancies. IT contractors played a significant role.
“It gives us a reach into technology…and innovation that we couldn’t develop in-house, but also gives us flexibility so we can go up and down on headcount if we need to during the year,” GCHQ told Parliament’s Intelligence and Security Committee.
Analysis by this committee in 2015/16 showed that the intelligence services had hired more than 1,000 external contractors through one classified managed services contract alone.
The contract added 10% to the number of people working for the security agencies.The cost of contractors was, on average, double that of internal employees.MI5 hired the majority of its hourly rated contractors, some 470 personnel, through this contract at a cost of £63m, an average of £134,000 per person.GCHQ hired 494 contractors at a cost of £71m, an average of £144,000 per person.SIS (MI6) hired 279 contractors at a cost of £40m, an average of £143,000 per person.
Source: Intelligence and Security Committee of Parliament Annual Report 2016-2017
The Soviet spy Kim Philby, whistleblowers such as Katherine Gunn – a GCHQ analyst who was threatened with the Official Secrets Act for disclosing an illegal attempt to bug members of the UN security council over the war in Iraq – and more recently Edward Snowden, show that vetting is no guarantee that intelligence agencies can keep sensitive data secure.
GCHQ has refused to confirm or deny whether it shares access to its intelligence databases with other members of the Five Eyes intelligence sharing group, made up of the US, New Zealand, Canada, Australia and the UK.
Lol! Nice line up.
True or false: Can sensitive data be accessed by command line interfaces?
GCHQ deputy director: Data on GCHQ’s data storage and retrieval platforms is not in a format that can be interpreted using the command line. The data is…hosted in such a way as to optimise the data for the analysis being carried out using the appropriate managed interface.
Security expert response: There are ways you can tweak data to make it quicker to search, for example GCHQ might hash phone numbers to make them quicker to search. To a degree he is probably right. You can go into a database and it could be hard to discern. But that does not stop you downloading large chunks of the database, and looking at it later in the database software.
GCHQ deputy director: A simple example would be a Microsoft Word document which if access via the command line returns a garbled set of characters because data needs to be placed through a Microsoft Office converter to present the information into a readable text format.
Security expert response: Computer Weekly witnessed demonstrations by two security specialists that showed it was possible to read the contents of a Microsoft Word file from the command line using a few simple commands. The complete text was visible, only the formatting and layout was lost.
GCHQ deputy director: The data needs to be stored in such a way to allow identification of a specific desired data item, i.e. a data item may not be stored in one place, rather being distributed across a number of storage servers, which can only be reassembled using specialist software.
Security expert response: GCHQ uses commercially available and open source databases, such as Hadoop and Elasticsearch, for managing large volumes of data. These programs have their own command line interfaces (CLIs), which can be accessed from a server command line interface. Systems administrators can use the software’s command line interfaces to perform the same operations as the database software, including searching for data. If anything, the command lines give systems administrators more powerful capabilities. It is possible that GCHQ may have decided not to install the command line interfaces that are built into commercial software. But it is unlikely. They get widely used for diagnostics.
“If I have a problem with a machine and I want to diagnose it by using the CLI for the database, I can check that the database is working. So while you could disable the database, you would be hampering systems administrators’ ability to troubleshoot,” one security expert told Computer Weekly.
GCHQ deputy director: The search tools available via the command line are basic, and considering the scale of data that needs to be searched against they would time out.
Security expert response: The claim is wrong. The time-out values will be the same for a web interface – i.e. the interface seen by a GCHQ analyst – as for the command line. If anything, the command line will be faster. Although CLI tools might appear basic, compared to a graphical interface, in the hands of a systems administrator, they are extremely powerful.
GCHQ deputy director: Although the technical community would state, in theory, that it is at times possible to search for a string – a name, for example – within the data using the command line, in practice this is not how the interface is used, nor is the interface designed to enable this kind of use.
Security expert response: This is inaccurate. It is possible for systems administrators to access the command line interfaces of database software and carry out searches. Computer Weekly saw a demonstration showing how a systems administrator carried out a search of fields in a Progress database held on a remote server, and was able to display the results.
GCHQ deputy director: Typically, with GCHQ the level of complexity of the systems means the only way to access the data in a readable format is via the software application programming interfaces (APIs).
Security expert response: A lot of databases, like Elastic Search and Hadoop, are store their data in JSON format. Using command line you could create your own record, or craft an API call from the command line. The response you get back is in a JSON format. A tool called JQ allows you to convert text into a JSON call. .
The files, from both the NSA and GCHQ, were obtained by the Guardian, and the details are being published today in partnership with the New York Times and ProPublica. They reveal:
• A 10-year NSA program against encryption technologies made a breakthrough in 2010 which made "vast amounts" of data collected through internet cable taps newly "exploitable".
• The NSA spends $250m a year on a program which, among other goals, works with technology companies to "covertly influence" their product designs.
• The secrecy of their capabilities against encryption is closely guarded, with analysts warned: "Do not ask about or speculate on sources or methods."
• The NSA describes strong decryption programs as the "price of admission for the US to maintain unrestricted access to and use of cyberspace".
• A GCHQ team has been working to develop ways into encrypted traffic on the "big four" service providers, named as Hotmail, Google, Yahoo and Facebook.
The papers are the latest to emerge from the cache leaked by the American whistleblower Edward Snowden, the former NSA contractor who has railed at the reach of the US and UK intelligence agencies.
Snowden warned about the relationship between the NSA and GCHQ, saying the organisations have been jointly responsible for developing techniques that allow the mass harvesting and analysis of internet traffic. "It's not just a US problem," he said. "They are worse than the US."
As well as the payments, the documents seen by the Guardian reveal:
• GCHQ is pouring money into efforts to gather personal information from mobile phones and apps, and has said it wants to be able to "exploit any phone, anywhere, any time".
• Some GCHQ staff working on one sensitive programme expressed concern about "the morality and ethics of their operational work, particularly given the level of deception involved".
• The amount of personal data available to GCHQ from internet and mobile traffic has increased by 7,000% in the past five years – but 60% of all Britain's refined intelligence still appears to come from the NSA.
• GCHQ blames China and Russia for the vast majority of cyber-attacks against the UK and is now working with the NSA to provide the British and US militaries with a cyberwarfare capability.
The details of the NSA payments, and the influence the US has over Britain, are set out in GCHQ's annual "investment portfolios". The papers show that the NSA gave GCHQ £22.9m in 2009. The following year the NSA's contribution increased to £39.9m, which included £4m to support GCHQ's work for Nato forces in Afghanistan, and £17.2m for the agency's Mastering the Internet project, which gathers and stores vast amounts of "raw" information ready for analysis.
The NSA also paid £15.5m towards redevelopments at GCHQ's sister site in Bude, north Cornwall, which intercepts communications from the transatlantic cables that carry internet traffic. "Securing external NSA funding for Bude has protected (GCHQ's core) budget," the paper said.
In 2011/12 the NSA paid another £34.7m to GCHQ.
The papers show the NSA pays half the costs of one of the UK's main eavesdropping capabilities in Cyprus. In turn, GCHQ has to take the American view into account when deciding what to prioritise.
A document setting out GCHQ's spending plans for 2010/11 stated: "The portfolio will spend money supplied by the NSA and UK government departments against agreed requirements."