Fragile systems make twits of us all

by John Naughton

The Observer

On Tuesday, April 23, a tweet from Associated Press (AP) revealed startling news. There had been explosions in the White House and Obama had been injured. The tweet was a hoax — the AP Twitter account had been hacked via a clever phishing exploit — but it briefly caused havoc. The Dow Jones Industrial Average dropped 144 points between 10.07 a.m. and 10.09 a.m., for example. Crude oil prices also briefly tumbled and the price of U.S. Treasury bonds and gold futures spiked. Within minutes, AP disclosed that the tweet was erroneous and things returned to normal, with the Dow eventually rising 152 points for the day to close at 14,719.

Crisis over, then? Er, not quite. The story of the hoax AP tweet resurrects troubling thoughts about systems and fragility. The first and obvious one is how it confirms the craziness of having computerized share-trading systems with hair-trigger settings. The frantic unloading of stocks that went on in the seven minutes before the hoax was uncovered was not done by human beings but by software programmed not only to read standard professional news feeds, but also to monitor the Twitter fire hose. Although software is good at lots of things, exercising judgment and common sense isn’t one of them.

Nobody knows for sure who was behind the hacking of the AP Twitter account, though the so-called Syrian Electronic Army claimed the credit. But the thought immediately arises that somebody could make a lot of money from a stunt like this — which is why we will see more of it in the future. As a commentator on the business news site Quartz put it, “Seven minutes is an eternity in the world of high-frequency trading, where equities are exchanged in fractions of a second. Trading bots could certainly have bought index funds at the bottom of the plunge, quickly profiting as the rest of the market realized that the AP tweet was wrong.”

But, in fact, the nervous volatility of computerized stock trading is just one part of a much bigger story, which is that we have been sleepwalking into total dependence on systems of terrifying fragility, without apparently realizing the potential consequences. Take the logistical systems without which none of our supply chains can now function. Supermarkets — even huge chains such as Tesco — carry only a few days’ stock: If the computerized trucking systems that ensure that they never run out were to be disrupted for even a few days, that — plus the resultant panic-buying — would effectively bring the country to a halt.

The strange thing is that these fragile systems were all built by people pursuing narrowly rational ends — to maximize efficiency and optimize the utility of company assets. In that sense, it’s all Toyota’s fault, because the Japanese car-maker was the company that introduced the idea of “just-in-time” manufacturing — the notion that holding large stocks of materials is wasteful, and it’s much better to organize production and services so that what you need is delivered just before you need it. This was the so-called “lean machine,” which came to dominate the economies of the entire industrialized world. It’s lovely when it works, but really scary when it fails. Just ask the British companies that found themselves screwed by the aftermath of the Japanese earthquake. I asked someone who worked in the motor industry what they had learned from the disaster. “That we need to know every single link in our supply chain,” he replied, “down to the suppliers of the suppliers of our suppliers.”

The hoax AP tweet has lessons for journalism too. The extent to which journalists have come to rely on Twitter is understandable: they’re a hyperactive breed. But somewhere along the line, some of them seem to be losing their judgment. Witness the torrent of vapid tweeting and retweeting — by journalists as well as members of the public — that accompanied the Boston bombings. This bears the same relationship to news as crab sticks bear to the real thing, which is why some seasoned hands, such as Dan Gillmor, have started campaigning for “slow news.” “We absolutely have to get it into our heads,” he writes, “that we can trust nothing at first glance. Nothing.”

Spot on. But it’s back to the future, really. I remember a time when a mantra of foreign correspondents was: “Never believe anything until it has been denied three times by the Quai d’Orsay [the French foreign ministry].” If we applied the same logic to our Twitter streams, we would definitely be less excitable, but we might also wind up considerably wiser.

Coronavirus banner