WASHINGTON/SAN FRANCISCO – Among defense contractors, QinetiQ North America is known for spy-world connections and an eye-popping product line. Its contributions to national security include secret satellites, drones and software used by U.S. special forces in Afghanistan and the Middle East.
Former CIA Director George Tenet was a director of the company from 2006 to 2008 and former Pentagon spy chief Stephen Cambone headed a major division. Its U.K. parent was created as a spinoff of a government weapons laboratory that inspired Q’s lab in Ian Fleming’s James Bond thrillers, a connection QinetiQ (pronounced kin-EH-tic) still touts.
QinetiQ’s espionage expertise didn’t keep Chinese cyber-spies from outwitting the company. In a three-year operation, hackers linked to China’s military infiltrated QinetiQ’s computers and compromised most if not all of the company’s research. At one point, they logged into the company’s network by taking advantage of a security flaw identified months earlier and never fixed.
“We found traces of the intruders in many of their divisions and across most of their product lines,” said Christopher Day, until February a senior vice president for Verizon Communications Inc.’s Terremark security division, which was hired twice by QinetiQ to investigate the break-ins. “There was virtually no place we looked where we didn’t find them.”
QinetiQ was only one target in a broader cyberpillage. Beginning at least as early as 2007, Chinese computer spies raided the databanks of almost every major U.S. defense contractor and made off with some of the country’s most closely guarded technological secrets, according to two former Pentagon officials who asked not to be named because damage assessments of the incidents remain classified.
As the White House moves to confront China over its theft of U.S. technology through hacking, policymakers are faced with the question of how much damage has already been done. During their multiyear assault on defense contractors, the spies stole several terabytes — equal to hundreds of millions of pages — of documents and data on weapons programs, dwarfing in sheer quantity any theft of Cold War secrets. The QinetiQ hack may have compromised information vital to national security, such as the deployment and capabilities of the combat helicopter fleet.
“The line forms to the left when it comes to defense contractors that have been hacked,” said James Lewis, a senior fellow in cybersecurity at the Center for Strategic and International Studies in Washington. “The damage has been significant.”
A few of the attacks have become public, including the 2007 theft from Lockheed Martin Corp. of technology related to the F-35, the most advanced U.S. fighter jet. Intelligence officials say the damage is far more extensive than the limited public accounting suggests, and that China-based hackers have acquired data on a large number of major weapons systems and many minor ones. One former intelligence official described internal Pentagon discussions over whether another Lockheed Martin fighter jet, the F-22 Raptor, could safely be deployed in combat, because several subcontractors had been hacked.
In 2007-2008, the Pentagon gave secret briefings to about 30 defense companies alerting them to the aggressive spying effort and providing data to help defend against it, according to a person familiar with the process. The person did not know whether QinetiQ received the classified intelligence.
Investigators eventually identified the Shanghai-based hackers that broke into QinetiQ as a crack team, nicknamed the Comment Crew by security experts, which has also hit major corporations and political figures, including the 2008 presidential campaigns of Barack Obama and John McCain. At least one other Chinese hacking team also may have been involved, according to a person familiar with the investigation.
In a Feb. 18 report, Mandiant, an Alexandria, Virginia-based security firm, attributed 141 major cyber-attacks to the Comment Crew without naming the targets. Mandiant identified the Comment Crew as the People’s Liberation Army Unit 61398, which is similar in some respects to the U.S. National Security Agency. Mandiant’s report prompted Tom Donilon, President Obama’s national security adviser, to call on China to stop the hacking of U.S. companies.
The spying on QinetiQ and other defense contractors appears aimed at helping China leapfrog the U.S.’s technologically-advanced military, forgoing years of research and development that would have cost billions of dollars, according to Michael Hayden, former director of the CIA.
China’s military may also have stolen programming code and design details that it could use to disable some of the most sophisticated U.S. weaponry.
The lengthy spying operation on QinetiQ jeopardized the company’s sensitive technology involving drones, satellites, the U.S. Army’s combat helicopter fleet, and military robotics, both already-deployed systems and those still in development, according to internal investigations. Jennifer Pickett, a spokesman for QinetiQ, declined to comment as part of a general policy not to discuss security measures.
“God forbid we get into a conflict with China, but if we did we could face a major embarrassment, where we try out all these sophisticated weapons systems and they don’t work,” said Richard Clarke, former special adviser to President George W. Bush on cybersecurity.
The spies’ trail at QinetiQ begins in late 2007, and so do the company’s mistakes. QinetiQ’s travails are documented in hundreds of unvarnished emails and dozens of reports that were never meant to be public, part of a cache that was leaked in 2011 by the group Anonymous after it hacked HBGary Inc., a Sacramento-based computer security firm hired by QinetiQ the previous year.
The emails and reports are authentic, according to former HBGary executives and Day. Day agreed to an interview limited to the investigation’s findings because the documents had already become public.
By reviewing the documents with security experts and interviewing more than a dozen people familiar with the QinetiQ breaches, Bloomberg News reconstructed how the hackers outmaneuvered QinetiQ’s internal security team and at least five companies brought in to help salvage the situation.
Headquartered in a glass-and-steel office tower in McLean, Virginia, QinetiQ’s U.S. subsidiary is a boutique arms maker, less than one-tenth the size of industry giants like Lockheed or Northrop Grumman Corp. It has specialized in fields expected to grow as the rest of the Pentagon budget shrinks, including drones, robotics, software and high-speed computing. A 2012 want ad for QinetiQ’s Albuquerque facility solicited a programmer to work on a “satellite-based global monitoring system” and limited candidates to those with top secret clearances only.
In December 2007, an agent from the Naval Criminal Investigative Service contacted the company’s small security team and notified them that two people working in McLean were losing confidential data from their laptop computers, according to an internal report. The agency had stumbled upon the stolen data as part of another investigation and the alert was a courtesy.
The San Diego-based agent didn’t provide the identity of the hackers, who had been tracked by U.S. intelligence since at least 2002, or the crucial — but classified — fact that they were hitting other defense contractors. The company wouldn’t find out who its attackers were for two more years.
QinetiQ put strict limits on the investigation.
“They just felt like it was this limited little thing, like they’d picked up some virus,” said Brian Dykstra, a forensics expert based in Columbia, Maryland, which QinetiQ hired to conduct the investigation.
Dykstra was given only four days to complete his work. He said the company didn’t give him the time or data necessary to determine whether more employees had been successfully targeted, a standard precaution. In his final report, Dykstra warned that QinetiQ “is likely not seeing the full extent” of the intrusion.
Evidence surfaced almost immediately that he was right, as the attacks continued. On Jan. 7, 2008, NASA alerted the company that hackers had tried to infiltrate the space agency from one of QinetiQ’s computers.
QinetiQ treated a series of attacks over the next several months as isolated incidents. The hackers followed a more meticulous strategy: In the first 2½ years, they gathered more than 13,000 internal passwords and raided servers that could give them detailed information about the company and how it was organized — data they would use to devastating effect.
More investigations uncovered more security holes. In 2008, a security team found that QinetiQ’s internal corporate network could be accessed from a Waltham, Massachusetts, parking lot using an unsecured Wi-Fi connection. The same investigation discovered that Russian hackers had been stealing secrets from QinetiQ for more than 2½ years through a secretary’s computer, which they had rigged to send the data directly to a server in the Russian Federation, according to an internal investigation.
QinetiQ’s executives in the meantime fretted about rising costs.
“You could spend all your resources chasing such things as this,” William Ribich, the former president of QinetiQ’s Technology Solutions Group, said in an interview in January. Ribich, who retired in November 2009, shortly after the discovery of a major data theft, said he needed to balance the uncertain risk that the hackers could use what they stole against a growing shopping list of security products and consulting fees.
“You finally have to reach a point where you say, ‘Let’s move on,’ ” he said.
China’s hackers in fact zeroed in first on Ribich’s division, based in Waltham, and specifically on QinetiQ’s drone and robotics technology. Internal reports leaked by Anonymous chronicle a breach at TSG in February 2008, followed by another attempt in March of that year. By 2009, the hackers had almost complete control over TSG’s computers, the documents show.
Over one stretch in 2009, the spies spent 251 days raiding at least 151 machines, including laptops and servers, cataloging TSG’s source code and engineering data. The hackers dribbled data out of the network in small packets to avoid detection, managing to get away with 20 gigabytes before they were finally stopped, according to an internal damage assessment.
The stolen cache included highly sensitive military technology and was equivalent in size to 1.3 million pages of documents or more than 3.3 million pages of Microsoft Excel spreadsheets.
“All their code and trade secrets are gone,” Phil Wallisch, senior security engineer at HBGary, wrote in an email after being briefed on the loss by the company.
It was about to get much worse.
While QinetiQ’s team tripped from crisis to crisis, the hackers honed their skills. They were next spotted in March 2010, after signing on with the stolen password of a network administrator based in Albuquerque, New Mexico, Darren Back.
The hackers logged on through the company’s remote access system, just like any employee. It was a trick they were able to use only because QinetiQ didn’t employ two-factor authentication, a simple device that generates a unique code employees enter, along with their usual password, anytime they work from home.
The problem had been spotted months earlier in a security review. Mandiant, which worked on several TSG breaches and performed the test, recommended a relatively inexpensive fix. The advice was ignored, according to a person familiar with the report.
In four days of furious activity, the hackers rifled at least 14 servers, taking particular interest in the company’s Pittsburgh location, which specialized in advanced robotics design. The Comment Group also used Back’s password to raid the computer of QinetiQ’s Huntsville, Alabama-based technology control officer, which contained an inventory of highly sensitive weapons-systems technology and source code throughout the company. The spies had got their hands on a map to all of QinetiQ’s digital secrets.
They also had begun to broaden their attack. As evidence mounted that the hackers had moved to divisions beyond TSG, QinetiQ hired two outside firms in April 2010 — Terremark and a relatively new startup called HBGary, headed by Greg Hoglund, a former hacker turned security expert.
HBGary installed specialized software on more than 1,900 computers, then scanned the machines for snippets of malicious code. Glitches surfaced almost immediately. The software wouldn’t load on at least a third of the computers, and even where it did, it missed some that the hackers’ spyware was known to have infected, according to internal HBGary emails.
Matthew Anglin, an information-security principal at QinetiQ, whose job was to coordinate the two investigations, fretted that he had no idea what was happening in his own network. He complained that the expensive outside experts didn’t seem to have a handle on what was going on, and wasted time tracing innocuous if unauthorized software.
The consultants also squabbled. HBGary complained in one report that Terremark was withholding vital information. Terremark countered that it appeared the hackers knew HBGary was hunting them and were using its technology to delete evidence of their presence on machines.
“They think we tipped off the attackers,” Wallisch, HBGary’s principal investigator on the project, wrote in an email.
The security teams found evidence that the hackers had burrowed into almost every corner of QinetiQ’s U.S. operations, including production facilities and engineering labs in St. Louis, Pittsburgh, Long Beach, Mississippi, Huntsville, Alabama and Albuquerque, New Mexico, where QinetiQ engineers work on satellite-based espionage, among other projects.
By the middle of June 2010, after weeks of intense work, the investigators believed they had cleaned QinetiQ’s networks and began wrapping up.
The calm lasted a little more than two months. In early September, the FBI called QinetiQ with evidence that the defense contractor was again losing data, according to emails and a person involved in the probe. Anglin messaged both HBGary and Terremark, asking how quickly their teams could return.
Within hours of their arrival, the investigators again began finding malicious software, or malware, in computers throughout the company’s North American divisions. Some of it had been there since 2009.
It began to dawn on the security teams that the hackers had established near a permanent presence in the defense contractor’s computers, mining new information almost as soon as it was written onto hard drives. “Oh yeah … they are f—-d,” Wallisch wrote to Hoglund in September.
Investigators also had to contend with frustrated QinetiQ employees. Upset about how much computer power the HBGary detection software was consuming, workers began deleting it from their computers with the approval of the company’s information technology staff.
As the hunt continued, more clues surfaced about what secrets the spies were after. The hunters’ digital footprints were found on the computers of QinetiQ’s chief operating officer, a division vice president and dozens of engineers and software architects, including several with classified clearances.
Among the victims was a specialist in the embedded software on microchips that control the company’s military robots, which would help in China’s own robot-building program, said Noel Sharkey, a drones and robotics expert at Britain’s Sheffield University. The PLA unveiled a bomb disposal robot in April 2012 similar to QinetiQ’s Dragon Runner.
The chip architecture could also help China test ways to take over or defeat U.S. robots or aerial drones, Sharkey said.
“You could set them up in a simulation board and hack into them,” he said. “That’s standard stuff.”
The spies also took an interest in engineers working on an innovative maintenance program for the U.S. Army’s combat helicopter fleet. They targeted at least 17 people working on what’s known as Condition Based Maintenance, which uses on-board sensors to collect data on Apache and Blackhawk helicopters deployed around the world, according to experts familiar with the program.
The CBM databases contain highly sensitive information including the individual PIN numbers of the aircraft, and could have provided the hackers with a view of the deployment, performance, flight hours, durability and other critical information of every U.S. combat helicopter from Alaska to Afghanistan, according to Abdel Bayoumi, who heads the Condition Based Maintenance Center at the University of South Carolina.
The hackers also may have used QinetiQ to break into the U.S. Army’s Redstone Arsenal through a network shared with QinetiQ’s engineers in nearby Huntsville. A breach of the base, home of the U.S. Army’s Aviation and Missile Command, was linked by military investigators back to QinetiQ, according to a person familiar with the investigation.
It wasn’t the only time the hackers used the same back-door approach to federal computers. The same person said that as recently as last year, federal agents were looking into a breach at a QinetiQ cybersecurity unit, which they suspected Chinese hackers were using in attacks against government targets.
The security lapses at QinetiQ led to investigations by several federal agencies, including the FBI, Pentagon, and Naval Criminal Investigative Service, according to two people involved, who didn’t know the final outcome of the probes.
The State Department, which has the power to revoke QinetiQ’s charter to handle restricted military technology if it finds negligence, has yet to take any action against the company. Two former federal law enforcement officials said that, despite its authority, the State Department lacks the computer forensics expertise to evaluate the losses and neither could recall department involvement in several major data theft investigations.
“In this case it looks like years go by without seeing any learning curve and that’s what’s scary,” said Steven Aftergood, who directs the Project on Government Secrecy at the Federation of American Scientists. “The company is responsible for its own failures, but the government is responsible for the inadequacy of its response.”
QinetiQ’s U.S. operations are overseen by a proxy board that includes Riley Mixson, the U.S. Navy’s former air-warfare chief. The board was briefed several times about the hacking and the investigations. In a brief telephone interview, Mixson said that “everything was duly reported” and then hung up the phone. Tenet declined to comment.
The investigations didn’t affect the company’s ability to win government contracts, even to provide cyber-security services to federal agencies.
In May 2012, QinetiQ received a $4.7 million cybersecurity contract from the U.S. Transportation Department, which includes protection of the country’s critical transport infrastructure.
“When it comes to cyber security QinetiQ couldn’t grab their ass with both hands, so it cracks me up that they won,” Bob Slapnik, vice president at HBGary, wrote after QinetiQ received a grant from the Pentagon in 2010 to advise it on ways to counter cyber-espionage.
In the fall of 2010, Terremark sent a report to Anglin concluding that QinetiQ had been targeted by the Comment Crew since 2007 and that the hackers had been operating continuously in their networks since at least 2009. The report was part of the trove of documents leaked by Anonymous.
In that time, the hackers had gained almost complete control over the company’s network. They had operated unhindered for months-long stretches and they had implanted multiple, hidden communications channels to extract data. Privately, the investigators concluded that the spies had gotten everything they wanted from QinetiQ’s computers.
“My feeling is that if an attacker has been in your environment for years, your data is gone,” Wallisch wrote in an email to a colleague in December 2010, a few weeks before HBGary itself was hacked and the record stops.
“Everything about your business is known, cataloged, analyzed, by your enemy,” Wallisch wrote. “I don’t feel a sense of urgency anymore.”