This is the same topic of Eliezer Yudkowsky's recent book "Inadequate Equilibria" [0], reviewed by Scott Aaronson here [1]. That book does a good job explaining why not every inefficiency can be eliminated - because getting out of a bad nash equilibrium requires coordinated changes, and for large enough groups you can't convince enough people to make a change at the same time.
Or you could read short book on game theory, think for ten minutes about perverse incentives, and get to the same results without the conceptual diarrhea that Yudkowsky insists on. Why he had to try to drag a concept of free energy into it with no justification, I don't know.
"Read a short book on..." Is a pretty high bar for conceptual simplicity.
Why wouldn't those who have had exposure to the concept of free energy prefer that metaphor to yours? You may be biased having exposure to more game theory than free energy. One person's conceptual diarrhea is another person's cleanest explanation.
>"Read a short book on..." Is a pretty high bar for conceptual simplicity.
Take five minutes and read the wikipedia page on the prisoner's dilemma then.
Even in a two player game, it's clear why individually optimizing for personal gain leads to poor outcomes for everyone, yet if there was a way to force people to act in a particular way, the total outcome for everyone would be better for all.
Except people tend to solve the prisoner's dilemma problem well enough that many research experiments lead to cooperative strategies. So it's a poor model of the real world. It's a dilemma for people who believe everyone's a rational optimizer.. for lack of a better word let's call them economists. There are variations of coordination problems that are actually complex enough to go unsolved in the real world and in some cases they don't even necessarily require irrational decisions, they just require a large amount of coordination to adjust the state to something where each individual decision has justified payoff.
"A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated."
There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing. This is not an accusation of danluu, though, as I believe it is an accurate statement of an error made by a lot of people.
Market efficiency is really a rather boring statement about the difficulty of performing arbitrage in the long term. Plus, given that the market is not absolutely efficient we should expect errors to be made.
But we can do some numerical analysis here. Consider the range of possibilities. Suppose population X is paid 0% of what everyone else is paid in the market, for the same output. It would take an astonishingly-large, an implausibly large, countervailing force to prevent some company from swooping in and claiming that free labor. (Remember, even if 99.9% of the firms on the market choose not to, it just means that the .1% is going have that much easier of a time.) However, for group X to be paid 99.9% of what everyone else is paid on average doesn't take much at all; indeed, mere noise could produce that result. Rather than it being a question of "is or is not discrimination", this sort of analysis can put numbers on how much, from which you can start analyzing things like the value of certain changes vs. their costs.
No conclusion; I'm just musing here. Except that I think that people generally do a lot of binary thinking here when quantitative might be more useful.
"There's equivocation in that statement; what market efficiency is and what people generally mean by efficiency are not the same thing"
True, but: it's no coincidence that the terms are used for different things. What we really care about as humans is "market efficiency" in the ordinary English sense of the word. That's really hard to define, though, so economists redefine it in a very precise, technical, and yes, 'boring' statement. Now it is easier to see if we have it, but also not very useful. However, it can still be used to argue that any intervention in the market will be a mistake. The confusion about what the term "market efficiency" means is not just a coincidence. Redefine it as something easily proven, and then let people mistake that for a proof of what they actually care about, and you get the same political/lobbying result as the (rather more difficult, perhaps impossible) task of proving that markets are efficient in the way we actually care about as humans, rather than just economists.
The definition is easy: by market efficiency, people mean time (labor market being the only market Adam Smith concerned himself and invisible hand being a reference to how the people themselves will tacitly protect their own interests).
And right now, the sense is that we're spending a lot of our time in jobs that overwhelmingly benefit a rich minority, which is not time spent well it.
When our ability to focus on self-preservation is being expropriated, people revolt
This interpretation isn't even up for debate, IMO. It's been investigated by biologists, neuroscientists, psychologists, medical doctors, philosophers, logicians, mathematicians, physicists, computer scientists.
A curious omission: economists. Very few of re-known willfully admit their own field is being twisted into undermining the majority for the minority.
The argument that xyz can't be the case because it would be arbitraged away is called the "invisible handwave". It should be part of the list of common rhetorical fallacies IMO.
Whether something will be subject to arbitrage is sensible to a ton of factors (barriers to entry, market structure, political environment/incentives, behavioral biases) that if something is to be arbitraged or not is purely an empirical question for any instance of the problem.
"It would take an astonishingly-large, an implausibly large, countervailing force to prevent some company from swooping in and claiming that free labor."
On the other hand, "no one had ever tried it" is an unbelievably strong force.
> A statement I commonly hear in tech-utopian circles is that some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated.
At the risk of sounding like an old fogey, I believe a similar statement can be made for stats-utopian circles and their belief in ability to quantify every thing.
As the post says:
> One nice thing about sports is that they often have detailed play-by-play data and well-defined win criteria which lets us tell, on average, what the expected value of a decision is.
So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.
Exactly. The biggest problem with quantifying efficiency in particular is that you have to define the inputs and outputs. Your biases are embedded in that definition rather than in the calculation per se. You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.
> You can make some pretty awful processes "efficient" if you assign zero value to some of the inputs, and conversely some pretty great processes "inefficient" if you assign zero value to some of the outputs.
I'm halfway convinced that time-motion studies are some kind of demonic invention for exactly this reason. We can get 3% more productivity out of an assembly line, just by doubling the accident rate and giving every worker RSI!
Goodhart's Law is dangerous even when we're making management and business decisions in good faith, but when ideas like "long term injuries are irrelevant" are implicitly added it becomes monstrous.
>So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.
Wouldn't it make his result even stronger? If the thesis fails to hold even in better conditions (like when there is a well-defined expected value), it should also fail to hold in worse conditions.
Using sports, particularly baseball is a terrible proxy, as the core assumption is that each game is a finite event.
That’s really not true at all — key decisions like whether to relieve a pitcher are not made within the context of a game. At a higher level, decisions to trade players are made that will affect outcome as well.
As someone who manages projects, there are parallels from a context point of view to be drawn to baseball. Each project is a measure of success or failure, but my effectiveness isn’t defined by individual projects. I may choose to do a project less efficiently to control cash flow or prioritize something else.
> So, using sports as a proxy is not the same as management decisions where there are no well-defined criteria or expected value.
Small aside but some including me would have a problem with even using sports as a proxy. There's a school of thought in soccer that maintains that the best match is a 0-0 draw [1][2][3]. While the data reflects a completely unproductive match, a 0-0 draw means that both sides performed admirably in defending and in countering the efforts of the other side. There is very detailed stats in soccer too now, but, that doesn't weaken the point.
Absolutely not. In sports, data-based, "rational" decisions are trivial. If poor decisions are made there, there is no chance that a decision in a less well defined field will not be hogwash.
This article is quite interesting for the isolated baseball analysis but is ultimately a long-winded discussion (which goes down a sports rabbit hole which ultimately doesn't contribute to the argument.) The TLDR is:
> If we want to look at the quality of decision making, it’s too simplistic to say that we expect a firm to make good decisions because they’re exposed to markets and there’s economic value in making good decisions and people within the firm will probably be rewarded greatly if they make good decisions.
Is there actually anything a reader here can take from this writing other than the assertion I quoted above (which is the author's point of view and isn't proven in nearly as much rigour as the baseball analysis is.)
I think the argument holds up: it is that, even in a field where management decisions can be clearly quantified, the data is collected, and people are doing the analysis, bad decisions persist (management makes bad decisions when they could make good decisions, and the market does not value good management decisions). So in a field where it is much less clearly quantified, why would we expect the market to ensure high-quality management decisions?
I don't think the post proves that the market fails to incentivize good decisions. That's a hard thing to prove. It does certainly give us serious reason to doubt the common claim that the market succeeds in incentivizing good decisions on a short timescale. (It might still be true, but we should have much less confidence in that statement being true than before we read this post.)
The claim "the market succeeds in incentivizing good decisions on a short timescale" is at odds with the article's demonstrated conclusion "the market fails to incentivize good decisions on a short timescale in the field of baseball". So you're left with two possible ways to resolve this: either baseball is unique and the market doesn't work properly there for reasons that don't generalize (but the article gives some reasons to believe that this isn't the right way to resolve the contradiction), or baseball isn't special and non-baseball markets very likely do similar things.
This looks super interesting. Unfortunately, I couldn't follow the explanation beyond the few first paragraphs because of so much baseball terminology and logic, which is completely alien to me :( The intro is confusing enough, but then there are terms that aren't defined, such as "stealing" - are readers supposed to be able to infer what this means just by context?
So the whole intro is to make the point that a certain type of play (base stealing) doesn't make sense, even for teams that do it well - and it took until 2012 to figure that out and get their players to stop doing that play.
Ceteris paribus, base stealing doesn't make sense. But baseball is a game played by humans and even if it weren't, there are a large number of variables that don't go into the simple analysis.
For example:
The members of the defending team adopt fielding positions based on a maximum potential for fielding a hit ball. But if there is even a threat of a stolen base attempt, the defense must change their positioning accordingly, affecting their ability to field a ball put into play.
If a player does make an attempt to steal a base, at least one defender must begin to change their positioning before the hitter has even had a chance to hit the ball. If they do not, they will not make it to the base in time to receive the throw from the catcher and either the throw will not occur or it will go out into the outfield and the runner may advance another base. Alternatively, the hitter may actually hit the ball and the ball may go to an area that it is statistically likely to go, where a defender would have been but for the stolen base attempt. What was to be an out is now a hit and the runner, having a head start because of their stolen base attempt, is able to advance further than they normally would have been able to. Because the hitter hit the ball, this situation does not get recorded in the statistics as a stolen base attempt!
Baseball is really a fascinating game to think about!
> For example: The members of the defending team adopt fielding positions based on a maximum potential for fielding a hit ball. But if there is even a threat of a stolen base attempt, the defense must change their positioning accordingly, affecting their ability to field a ball put into play.
Yeah, that was the biggest flaw I spotted in the analysis of base-stealing: it doesn't seem to capture effect of the threat of base stealing on the defense's behavior, which is very probably non-zero and in favor of the offense. If you never, ever steal, and the defense knows that (someone'll notice before long), they can play better defense against the batter. It may still work out that a never-steal rule still provides better outcomes (by a smaller margin), but I'm guessing "rarely steal" ends up being the better guideline, overall.
If that's true, then I believe it means that the defenders are overcompensating for stealing. As I understand it, in equilibrium you'll have the feature that the defense cannot benefit by guarding steals less closely in exchange for better fielding position.
If trying to steal loses bases, then the defense could spend less effort preventing steals, and more effort fielding.
This principle comes up a lot: if you get into every college/job you applied to, then you probably didn't apply to enough schools/jobs, unless 1) you didn't want to go to Harvard/AppAmaFaceGooSoft, 2) you got in to Harvard/AppAmaFaceGooSoft, 3) you couldn't afford the applications.
He briefly addressed this in one of the appendices:
"Another reason to sac bunt (or bunt in general) is that the tendency to sometimes do this induces changes in defense which make non-bunt plays work better."
It apparently changes the probabilities slightly, but not enough to change the overall conclusion.
Yes, I would expect that a drop in stolen base attempts would lead to a drop in errors by catchers.
Looking at stats from 2000-2017, total errors underwent a steady drop from ~3400 in 2000 to ~2800 in 2017.
On the other hand, it is not entirely clear that teams stopped stealing bases starting in 2012. Yes, 2015-2017 were far below average, but so were 2003-2005.
A lot of those moneyball decisions seem to have very marginal impacts, and I feel like the differences in team quality across the league were large enough that those subtle statistical advantages didn't matter very much. Moneyball only became a big thing after the luxury tax was implemented when a lot of teams were basically at parity in terms of player ability and small tactical advantages suddenly became meaningful.
There are nine innings, during half of each of which one team plays offense and the other plays defense. They switch for the remaining half of each inning.
The offense is permitted three "outs" in their half of each inning (after which the teams switch roles, or the next inning commences).
There are four bases, the first three of which may be occupied by a member of the offense. Bases are ordered. Players may advance directly from one base to the next by "stealing". This comes at the risk of an out, which results in losing occupation of a base entirely.
A player which reaches the fourth base scores a "run" for the offense and no longer occupies a base. Runs (and only runs) count toward the final score.
One player from offense is always "at bat". That player has a chance to occupy a base (not necessarily the first); failure to do so results in an out. During this process, the other base occupants may (be forced to) advance bases, possibly scoring runs or losing bases and gaining outs in the process. (The particulars of this are not relevant to the article, but there is a fun story about the process here [1].)
Rather than permit the player at bat the chance to advance potentially several bases, the defense can "walk" the batter by granting them uncontested occupation of first base.
Similarly, rather than permit the defense to prevent any base advancement, the player at bat can choose a "sacrificial bunt" ("sac" bunt) which, properly executed, nearly guarantees that players currently occupying bases advance one base (or even score), while the player at bat (and only that player) is called out (and thus does not advance). SPOILER ALERT. This action is key to the plot in a particular episode of Deep Space 9 [2].
An aside to the main point: the author claims that one can do a similar analysis to other sports. My intuition is that you can't do quite as accurate an analysis of, say, soccer, than you can baseball. As a sport, baseball is unusually discretized when compared to a much more continuous sport like soccer or hockey. Being discrete, it makes it much easier to represent as a state machine, which makes it easier to simulate, and easier to suss out cause and effect. American football, I think, may be somewhere inbetween. I know, of course, that other sports do significant statistical analysis, but my intuition is that it is harder to attribute cause and effect at the fine-grain level that one can with baseball.
From what I can gather making good management decisions is a lot like making a good estimate for a software project. There are two ways to go about it: (A) do a lot of work to break down the problem to manageable parts, put a well-reasoned number on each part, then sum up the parts with the necessary tolerances and risk factors, or (B) just go with your gut.
Speed is very important in business. How much time would it take to estimate how much time it would take to analyze and estimate the management decision in question?
What is gut if not an accumulation of experiences? If an engineering-manager-with-10-years-experience's gut produces the same accuracy as a fresh grad's, then it's not of much use, and there's a good chance that the manager would generate a bad estimate even following (A). That's not to say that people shouldn't analyze. Just saying that gut is not random guesswork.
I feel like 80/20 Rule works here. Describe the why, what, and 20% of the how that answers 80% of how to do what. The other 80% that answers the last 20% should rest on employee. That way you are not micromanaging and give way too human creativity.
One issue with analysis like this is micro optimizations have more information than simplified abstractions. Consider, in the bottom of the 9th 1 or 2 runs may have identical value. So, reducing the total expected value of runs to increase your odds of a single run may be a very good trade off.
And that's assuming you want to optimize total wins. Increasing your odds of winning the playoffs may reduce your odds of getting into them by having more higher paid people, but a lower average player. Further, maximizing making money over time is yet another option, which may promote showmanship over success.
"But there are situations where the approximation isn’t very good, such as when it’s the 9th inning and the game is tied. In that case, a decision that increases the probability of scoring 1 run but decreases the probability of scoring multiple runs is actually the right choice."
I understood that the "full" model includes these considerations.
I was bringing up that edge case specifically because 'winning' is the obvious optimization, but not necessarily the correct one.
Bottom of the 9th bases are loaded and the guy at bat hit's a home run, that's the kind of thing that sticks with people and makes more money in the long run than a bunt that get 1 run.
All companies make both good and bad decisions. They don't fall apart when they make a single bad decision. They fall apart when he sum total of the impact of their bad decisions outweighs the sum total of the impact of their good decisions.
When this happens, the last bad decision usually takes the blame, however it is rarely the true cause.
>A less contentious example is that when you see a big company doing something that seems bizarrely inefficient, maybe it’s not inefficient and you just lack the information necessary to understand why the decision was efficient.
This pretty much sums up a lot of tech news and the associated comments on such news.
I've long said that if the way someone's behaving doesn't make sense, you don't understand their motivations.
Now, it may be that their motivations are all messed up and they're acting the way they are because sabotaging their relationships reaffirms their core belief that they're unlovable. But it does make sense once you understand the framework they're working in.
The problem is externalities -- both positive and negative externalities affect the outcome and it is basically impossible to determine whether an outcome occurred due to management decision or due to an externality (eg: revenue growth due to successful execution or revenue growth due to changing business cycles).
1. "Quality" for management decisions are notoriously ill defined for the same reason that we never know if we did the right thing with the economy: the experiments can never be repeated.
2. The example serves to explain what a decision is and what a good decision would be, but that is not the concept that needs explanation!
3. "some seeming inefficiency can’t actually be inefficient because the market is efficient and inefficiencies will quickly be eliminated." is not the right statement of efficient markets. Rather, the act of trying to make it more efficient is exactly what makes it efficient. So if you see an opportunity, someone exploits it and thus makes it more efficient.
> what a good decision would be, but that is not the concept that needs explanation
It is subsumed in this work that good for baseball is winning games, and if you look at some of the references, it is taken for granted that there are ways to rank the possible results on some scale of efficiency or effectiveness. I would certainly question this.
Some of the references study the huge money-eating monster: healthcare. Here is a simple non-controversial problem -- look at all the healthcare systems in the world and rank the 10 best in order from best to worst by efficiency, effectiveness or ???
Even in baseball, there is no way to rank results. Some years, some teams have had very low winning percentages but much higher profitability than in other years. When I studied economics, some very good economists like Vernon Smith and Roger Noll took an interest in sports and particularly baseball. A standard assumption then was that there was an optimum winning percentage at which game attendance would be maximized, around 60% to 70%.
Baseball is a good example of the difference between short-term and long-term objectives, too. The winning vs losing equation is viciously zero-sum, and almost any innovation will be copied if it succeeds in the short-run. This leaves about zero long-term incentive to innovate. The real questions are: What kind of game do you want to play? What kind of game do people want to watch? What kind of job do you want to have? What kind of relationships do managers want to have with their unwashed workers? Which stakeholders will the firm respect?
4. Markets aren't actually efficient because they are stacked in favor of incumbents through their ability to lobby for anti-competitive legislation, as can be seen with the perversion of the copyright and patent systems.
Markets aren’t going to deliver best value to anyone without being forced to.
Eventually, every company wants to own the market, and convincing government to do so is a key part of that strategy. The invisible hand sometimes is holding a visible club.
I am saying the opposite: the market becomes less efficient as a result of legislative capture. But it is a feedback loop, so attributing the dynamics to simple cause and effect is not appropriate.
You could replace that entire post with "No. Unlike baseball."
Not that the post wasn't interesting, of course. To me, it leads to the inevitable conclusion that we need to record failures as well as wins in order to get accurate data on management decisions.
0: https://equilibriabook.com/toc/ 1: https://www.scottaaronson.com/blog/?p=3535