A few weeks back, I asked whether online ads do any good at all. I cited a paper that had done experiments on the effectiveness of Google ads. Jim Manzi, who runs a whole lot of experiments for companies, writes to offer his perspective:
"Megan McArdle laments that online journalism may be going the way of print journalism. In doing so, she cites a research paper for a series of randomized field experiments at eBay showing that a set of paid search buys had causal effects on sales that were a small fraction of those estimated by nonexperimental methods, and that in certain cases had no measurable effect on sales at all.
"I am always heartened to see controlled experiments take some of the air out of claims derived from nonexperimental methods. The eBay experiments seem to have been well-designed and analyzed, but I don't think the implications of this are as broad as it might seem.