Prediction Machines

One of my favorite books of the dotcom era was Information Rules, by Shapiro and Varian in 1998. At the time, tech boosters were saying that all the old business rules were obsolete, and anyone who disagreed “just doesn’t get it.” But Shapiro and Varian showed in detail how to understand the new internet economy in terms of standard economic concepts. They were mostly right, and Varian went on to become Google’s chief economist.

Today many tout a brave new AI-driven economic revolution, with some touting radical change. For example, a widely cited 2013 paper said:

47% of total US employment is in the high risk category … potentially automatable over … perhaps a decade or two.

Five years later, we haven’t yet seen changes remotely this big. And a new book is now a worthy successor to Information Rules:

In Prediction Machines, three eminent economists recast the rise of AI as a drop in the cost of prediction. With this single, masterful stroke, they lift the curtain on the AI-is-magic hype and show how basic tools from economics provide clarity about the AI revolution and a basis for action by CEOs, managers, policy makers, investors, and entrepreneurs.

As with Information Rules, these authors mostly focus on guessing the qualitative implications of such prediction machines. That is, they don’t say much about likely rates or magnitudes of change, but instead use basic economic analysis to guess likely directions of change. (Many example quotes below.) And I can heartily endorse almost all of these good solid guesses about change directions. A change in the cost of prediction is a fine way to frame recent tech advances, and if you want to figure out what they imply for your line of business, this is the book for you.

However, the book does at times go beyond estimating impact directions. It says “this time is different”, suggests “extraordinary changes over the next few years”, says an AI-induced recession might result from a burst of new tech, and the eventual impact of this tech will be similar to that of computers in general so far:

Everyone has had or will soon have an AI moment. We are accustomed to a media saturated with stories of new technologies that will change our lives. … Almost all of us are so used the the constant drumbeat of technology news that we numbly recite that the only thing immune to change is change itself. Until have our AI moment. Then we realize that this technology is different. p.2

In various ways, prediction machines can “use language, form abstractions and concepts, solve the kinds of problem now [as of 1955] reserve for humans, and improve themselves.” We do not speculate on whether this process heralds the arrival of general artificial intelligence, “the Singularity”, or Skynet. However, as you will see, this narrower focus on prediction still suggests extraordinary changes over the next few years. Just as cheap arithmetic enabled by computers proved powerful in using in dramatic change in business and personal lives, similar transformations will occur due to cheap prediction. p.39

Once an AI is better than humans at a particular task, job losses well happen quickly. We can be confident that new jobs will arise with a few ears and people will have something to do, but that will be little comfort for those looking for work and waiting for those new jobs to appear. An AI-induced recession is not out of the question. p.212

And they offer a motivating example that would require pretty advanced tech:

At some point, as it turns the knob, the AI’s prediction accuracy crosses a threshold, changing Amazon’s business model. The prediction becomes sufficiently accurate that it becomes more profitable for Amazon to ship you the goods that it predicts you will want rather than wait for you to order them. p.16

I can’t endorse any of these suggestions about magnitudes and rates of change. I estimate much smaller and slower change. But the book doesn’t argue for any of these claims, it more assumes them, and so I won’t bother to argue the topic here either. The book only mentions radical scenarios a few more times:

But is this time different? Hawking’s concern, shared by many, is that this time might be unusual because AI may squeeze out the last remaining advantages humans have over machines. How might an economist approach this question? … If you favor free trade between countries, then you … support developing AI, even if it replaces some jobs. Decades of research into the effect of trade show that other jobs will appear, and overall employment will not plummet. p.211

For years, economists have faced criticism that the agents on which we see our theories are hyper-rational and unrealistic models of human behavior. True enough, but when it comes to superintelligence, that means we have glen on the right track. … Thus economics provides a powerful way to understand how a society of superintelligent AIs will evolve. p.222

Yes, research is underway to make prediction machines work in broader settings, but the break-through that will give rise to general artificial intelligence remains undiscovered. Some believe that AGI is so far out that we should not spend cycles worrying about it. … As with many AI-related issues, the future is highly uncertain. Is this the end of the world as we know it? not yet, but it is the end of this book. Companies are deploying AIs right now. In applying the simple economics that underpin lower-cost prediction and higher-value complements to prediction, your business can make ROI-optimizing choices and strategic decision with regard to AI. When we move beyond prediction machines to general artificial intelligence or even superintelligence, whatever that may be, then we will be at a different AI moment. That is something everyone agrees upon. p.223

As you can see, they don’t see radical scenarios as coming soon, nor see much urgency regarding them. A stance I’m happy to endorse. And I also endorse all those insightful qualitative change estimates, as illustrated by these samples:

When prediction is cheap, there will be more prediction, and more complements to prediction. p.15

From a statistical perspective, data has diminishing returns. … [However, sometimes] the additional data allows the performance of the prediction machine to cross a threshold from unusable to useable, or from below a regulatory performance threshold to above, or from rose than a competitor to better. p.51

One major benefit of prediction machines is that they can scale in a way that humans cannot. One downside is that they struggle to make predictions in unusual cases for which there isn’t much historical data. Combined, this means that many human-machine collaborations will take the form of “prediction by exception.” p.67

While prediction is a key component of any decision, it is not the only component. The other elements of a decision – judgment, data, and action, remain, for now, firmly in the realm of humans. p.76

AI can change work flows in two ways. First they can render tasks obsolete and there remove them form work flows. Second, they can add new tasks. p. 127

Our process … involved evaluating entire work flows, whether they are within or across jobs (or departmental or organizational boundaries), and then breaking down the work flow into constituent tasks and seeing whether you can fruitfully employ a prediction machine in those tasks. They, you must reconstitute tasks into jobs. p.142

In the AI world Google is Iowa. .. When the conditions were right, all corn farmers in Wisconsin, Kentucky, Texas, and Alabama eventually followed their Iowa peers in adopting hybrid cord. The demand-side benefits were high enough, and the supply-side costs had fallen. Similarly, the costs and risks associated with AI will fall over time, so that many business not at the forefront of developing digital tools will adopt it. p.160.

Your past data on yogurt sales has little value once you have a prediction machine built on it. In other words, it may be valuable today, but it is unlikely to be a source of sustained value. p.163

AI can lead to a strategic [business] change if three factors are present: (1) there is a core trade-off in the business model (e.g., shop-then-ship versus ship-then-shop); (2) the trade-off is influenced buy uncertainty; and (3) an AI tool that reduces uncertainty tips the scales of the trade-off so that the optimal strategy changes from one side of the trade to the other. p.165

Supposed that AI allows airlines not only to forecast weather events but to generate predictions for how bst to deal with weather-related interruptions. … [then] the major airlines … would require less capital equipment because they could outsource more flights to the smaller regional carriers. p.170

AI will shift HR management toward the relational and away from the transactional. … Employee contracts need to be more subjective. … Better prediction increases the uncertainty you have over the quality fro human work performed: you need to keep you reward function engineers and other judgment-focused workers in house. p.173

If the precision machine is an input that you can take off the shelf, then you can treat it like most companies treat energy and purchase it from the market, as long as AI is not core to your strategy. In contrast, if prediction machines are to be the center of your company’s strategy, then you need to control the data to improve the machine, so both the data and the prediction machine must be in house. p.177

There are no easy ways to overcome the trade-off that arise when precision alters crowd behavior, thereby denying AI of the very information it needs to form the correct prediction. p.191

In effect, experience is a scare resource, some of which you need to allocate to humans to avoid deskilling. … But if you put a human in the loop, how will that machine’s experience emerge? p.193

This is not the first time that a new technology raises the possibility of breeding large companies. … With AI, there is a benefit to being big because of scale economies. However, that doesn’t mean that jost one firm will dominate or that even if one dominates, it will last long. On a global scale, that its even truer. 215

We still don’t know if the scale advantage of AI is big enough to give Google an advantage over other large players. … There is not easy way to determine if the largest AI companies will get too big and no simple solution even if they do. … Breaking up monopolies reduces the scale, but scale makes AI better. Again, policy is not simple. p.217

Added 24Apr: I should have said: the book is more specifically about Mass Prediction Machines. Machines that automate predictions that we do often, and so have a lot of supporting data. These machines don’t help much with niche predictions, done rarely and without much supporting data.