Statistics is ultimately counting, and therefore is incredibly vulnerable to discretion in choosing "what counts". Take the unemployment rate as one example. When people realize that its not equivalent to "people who do not have a job", how can you complain that they trust statistics less?
Expert authority is in decline because it should be, as there is an increasing body of evidence that experts, from politics to medicine, have almost no advantage in forecasting power than the average person.
Why should "experts" (often just pundits) have any authority when they have consistently demonstrated they deserve very little?
Finally, the political slant of this article, going along with the decried "fake news", blaming the election results on these declines in authority, is pathetic. It's basically an extension of "the other side is filled with stupids" and has no credibility, no matter how you dress it in professional journalistic veneer.
Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous. Saying we should just give up on using data to make predictions is ludicrous -- even if it hasn't worked before in some situations.
We should hopefully be in agreement that using data to make decisions is preferred to winging it. We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways.
The thrust of your argument is valid -- summarizing data inherently leaves things out. However, the solution to this is not to give up and throw statistics out the window. And the solution is definitely not to unfairly discredit people who devote their lives to these issues.
Rather, if you see statistics, wonder how they were generated and why some data was included and others thrown out. By being open and candid about the decisions made, the statistics are meaningful. Just giving numbers without providing details of how they were achieved is not valid statistical work.
From my experience, most scientists (and myself in my work) care about the truth over everything else. I use statistics to better understand my data. My objective is always to choose the most sensible path in analyzing it that allows me to most accurately answer the questions I pose.
> We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways.
I have to admit that I don't trust social scientists with anything they say about social science. There have been too many replication failures. I don't think you are entitled to people's deference just because you spend your life doing something, there has to be something more "material" than that. If I spent my life trying to get a girlfriend would you go to me for dating advice?
Physics is one thing - you can easily demonstrate the atomic model is correct by building a nuclear fission generator. Computer Science is pretty objective, math.. But almost any other field my reaction is - "okay, well show me, use it". I'm not in the wrong there, you're in the wrong for not effectively proving it to me. Our default should be skepticism.
> Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous.
I think that "experts with agendas that supersede the truth" claim is true if you consider the pool of pundits that regularly appear in mainstream media. Unfortunately they are not representative of scientists like yourself.
This attitude of not trusting experts and not trusting statistics unfortunately permeates from the political arena into science.
The belief that all statistics are not to be trusted is a clear danger to our society. Not trusting climate scientists. Not even trusting simple statistics on vaccinations. As another poster noted, the only alternative is ignorance.
Exactly. So what we need to do is reduce the misuse and cherry picking in the statistics that make it to the main stream media (both sides) so that public confidence in science does not degrade further.
Even then the claim is not true, because then you would be wrong to call them "experts". The fault then lies not with the experts, but with the media you choose to watch.
It would be wrong by our standards to call them "experts", but that won't stop the media from using the term.
Personally I avoid news as much as possible, but it's almost impossible to avoid it totally (break rooms, visits to friends homes, etc mean that I still see a bit of mainstream news media).
Well most people don't do a complete study before choosing a media. They usually just turn on TV. I think that major media companies should be more responsible of what they say. Or how do it correlate to the life of people (like for the GDP).
"most experts have agendas that supersede the truth"
I spend a lot of time with education statistics, and this is absolutely commonplace. Most of it is spin; folks choose the metrics that support the point they want to make.
> We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways.
Unfortunately this is sometimes untrue. For example: Long Term Capital Management was run by some of the most expert academic financiers perhaps ever gathered together and it blew up. Lord Kelvin thought X-Rays were a hoax. It is the blanket assumption that specialists in a field should be trusted that is mistake. Specialism has an important place but epistemology is a difficult subject and it is a matter for everyone.
It's all about who to trust. Science sometimes gets it wrong for a while, as do some scientists. However, I would trust expert academic financiers over someone who by luck happened to have a good year with his or her stock picks.
> Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous
Are you talking about actual experts, or claimed experts?
The issue here is the high number of the latter indistinguishable from the former.
> Saying we should just give up on using data to make predictions is ludicrous
Who is "we". Statistics can still be performed, but it's unrestricted "authority" must be examined - it's too easy to do "bad" or "not" statistics, and claim it to be valid without restriction. It's also easy and common to use misleading statistics too, so simply outlawing bad statistics isn't enough - there need to be clear guidelines on non-ambiguity, much like exists for legal documents.
> I use statistics to better understand my data
You don't benefit from statistics half as much as those that use it to gain political power.
Sometimes ignorance isn't even an option: it's mandatory. The only choice we are left with is whether to admit to ourselves that we are ignorant.
The choice isn't between governing peoples lives using statistics vs. governing peoples lives based on gut-instinct (or whatever). It's between using weak statistics as a pretext for governing peoples lives vs. just admitting there's a lot we don't know and choosing to give people space to figure out things for themselves.
Jim Manzi, who literally built his business off of effective use of statistics, spent 10 years figuring out where to put the snickers bar at the checkout stand[0] and he admitted it was a really hard problem. Yet it's a joke compared to some issues that experts claim they can solve.
I have to depend upon statistical reasoning, both directly and indirectly, for work. I understand and appreciate the validity of the methods of statistics. That doesn't mean I'm going to trust any statistics that are used to back a policy proposal.
There are plenty of alternatives to statistics and expertise, as the sibling comment points out, letting people get on with things themselves (i.e. the default action of do nothing) is essentially a delegation to the wisdom of the crowds. Gut feeling and intuition are also alternatives people frequently rely on, and aren't necessarily nonsense: the brain is very good at synthesising conclusions at the level of the subconscious that surface consciously as hunches or gut feelings.
The "body of evidence" that self-proclaimed experts are frequently wrong is rarely presented statistically, but it could be so. However this would be circular, it'd be an expert using statistics to claim that experts using statistics can't be trusted. That's a time when 'common sense' knowledge is sufficient: if you've seen endless stories in the press about how experts predit X, and none about experts predicting not X, then when X doesn't happen you can safely lower your trust in experts without needing an Excel spreadsheet to prove it to yourself.
> Gut feeling and intuition are also alternatives people frequently rely on, and aren't necessarily nonsense: the brain is very good at synthesising conclusions at the level of the subconscious that surface consciously as hunches or gut feelings
And thus we get anti-vaxxers and climate change denial.
BLS is quite up-front about what the "unemployment rate" measures, and provides six different numbers using different definitions in order to better capture the big picture. I don't see why I can't complain if people's reaction to this information is to trust statistics less, rather than spend thirty seconds reading about where the numbers come from.
One argument I've seen is that although BLS provides many metrics, they consciously avoid several useful metrics that paint a realistic but negative picture of the economy. If I understand it, one big complaint is that "long term discouraged workers" are dropped from U6, and BLS does not publish a metric showing total labor force participation. I don't know enough about the details to assess know if this is true, but I thought this piece seemed like an objective criticism: http://www.shadowstats.com/article/c810x.pdf
That seems like a fair criticism. But at the same time, as long as they don't pretend that the numbers say more than they do, I don't see a big problem with them.
Most people who have recently been to a pro baseball game know that there are times to mistrust statistics. The home team likes to give attending fans something to look at during the long periods of relative downtime. For each hitter who comes up, they like to come up with a statistic that favors the home team to display on the screens. There are lots of options, so they can take their pick, even if they sometimes come up with things like "At-bats on a Tuesday against teams out-of-division".
However, everyone who's trusted any technology that accounts for quantum effects (e.g. modern computer processors) has trusted statistics. Everyone who's ever received a modern medical treatment (of any kind) has trusted statistics.
The hard part is knowing when you should, and when you should not, trust statistics. Frankly I don't think there's anyone who can do that 100% of the time -- but I think most people could probably learn to do well enough to benefit. The learning part is tough, though.
Is the problem that statistics are losing authority, or that everything connected with the elite is losing authority? From what I see, just about every class of experts is less trusted now than a few years ago. Economists, scientists (climate change), doctors (vaccines), and so on. Statistics are used to back up every quantitative thing that's ever been studied, so of course that quantitative element sometimes comes under scrutiny as well.
As for statistics as a national health monitoring system, it is a victim of regime shift.
You collect the statistics that seem relevant when you are setting up the collection. You look at things like money supply, unemployment, GDP and manufacturing jobs because they seem to make sense at the time. Whatever models people are working on seem to want those numbers, so those are the numbers you collect.
Unfortunately, it's hard to model things for which you have no data. Social science is especially messy, and so there are many ways things might interact, but haven't yet, and some of those things will be a surprise to you, with the added problem that the confounding variable is one that you haven't collected, because you didn't think it was important.
This regime shift, which causes a new way for the numbers to be generated, is to be expected. Why would the economy grow for hundreds of years, like it has, in the same way? Looking at history, it hasn't. The structure of the economy changes, as you would expect from any ecology that isn't in equilibrium. And "The World Economy" has been growing for hundreds of years.
One example of numbers changing meanings was in the Economist the other day. Manufacturing is not what it was. A lot of builders of equipment now make money maintaining equipment or training people to use it. For instance I was surprised to find out a professional coffee making machine is actually not just a thing a coffee shop buys. They lease it, and along with the lease comes training for baristas. Something that would skew figures.
The article doesn't mention the problem that we often try to predict the outcome of a singular event when statistics can only tell us something about it when repeating it. The prediction could still be correct even when the actual event is not what was predicted.
There's also something to be said for the replication crisis in certain highly-politicized fields of sociology and psychology.
People (rightly, IMHO) realize that political activists sometimes work under the guise of scientific researchers. It shouldn't come as a surprise that this kind of conduct erodes confidence in the field as a whole.
Sadly, the baby has a nasty habit of being thrown out with the bath water...
Not disagreeing with you, but the replication crisis is happening everywhere, especially in the biomedical sciences. In fact, analyses have suggested it may be worse in certain fields, like cognitive neuroscience (whether or not you consider that a branch of psychology or biology is maybe debatable).
This is a key problem fuelling climate change skepticism. People look at the structure of academia and notice that the way to get money in such a system is by convincing committees of your peers that your research is more important than other people's research. And saying "my research could literally save the world from total destruction" is the trump card in such a game, thus people are highly incentivised to play it.
That, combined with the facts that humans have a poor intuition for probability and statistics and some results in the field seem inherently paradoxical (if you told a lay person that zero probability events happen all the time, they would look at you like you are crazy).
It isn't, actually. There is a difference between discrete and continuous probability distributions. Lotto numbers are a classic example of a discrete, finite space, and the probability of winning the lottery can be expressed precisely as a non-zero number. The probability of any individual event in a continuous probability distribution is exactly zero. You're free to dispute whether the continuum as a mathematical construct has validity when applied to the world of actual stuff, but that's a different debate and certainly doesn't have an obvious answer.
It's not a different debate, it's an essential requirement for zero probability events to actually happen. For zero probability events to happen the observable universe needs to contain infinite bits of information. The true upper bound is subject to debate but it's widely believed to be finite (eg. below the Bekenstein bound). If it is finite, then by the pigeonhole principle, no method of sampling a continuous distribution can choose between more points from that distribution than can be enumerated by that many bits. This means the probability of making any specific selection is non-zero, because almost all of the distribution cannot be selected.
Pick a random real number in the range [0,1]. Because there are an infinite amount of such numbers, the probability of picking any specific real number is precisely 0. However, you will still end up picking some number.
Picking a random real number in the range [0,1] is impossible. The expected length of its simplest description is infinite, and if you cant describe it you haven't actually picked it.
Assuming you meant [0,1], I don't think it's accurate to describe infinitesimals like lim->0 as being equal to zero. Adding an infinite number of infinitesimals gives a nonzero result, but adding an infinite number of zeros does not.
Standard probability theory does not have a notion of infinitesimals.
There have been non-standard attempts at probability theory that do involve such a notion. See [0] (found on a brief search; I have only skimmed).
EDIT: Admittedly, the existence of the non-standards analysis is a good indication that this is an artifact of the formalization of probability theory; not of the underlying reality that we are trying to model.
Simpson's paradox is probably a better example of a "real" "paradox" of probability. [1]
While I don't disagree with what your saying, I would point out if you want to use specific definitions they don't apply in the general case.
Abstractly, there must be some difference between 0.2 and 2 as the first is a valid output and the second is not. Otherwise when you sum [0.2,0.3] vs [2,3] you get the same probability of 0.
But, there is a trade off that can make things a little cleaner if you define the probability at 0.2 to be 0.
> The prediction could still be correct even when the actual event is not what was predicted.
This is true (and conversely, a prediction can still be "wrong" even if the outcome matches the prediction, if the prediction expresses the wrong degree of confidence).
However:
> when statistics can only tell us something about it when repeating it
This is only true if you take a pure frequentist approach to probability. The Bayesian interpretation of probability has no problem with defining probabilities over single-occurrence events. (And even frequentists have a way of dealing with this, although Bayesians would argue that in doing so they're essentially adopting a form of Bayesian probability).
People often fail to trust what they don't understand, and instead believe those who offer easy to understand but often false or inaccurate information. People also look locally but are told things that apply globally and they can't connect the two. This of course means they are likely to be manipulated by people who know how to tailor the message. I don't know if our education system is up creating a population that can understand what they are being told and when they are being lied to. Life today is filled with so many decisions and concepts and information and connections and maybe it is too complex for most people to comprehend. This may just lead people to give up and never get beyond easy answers.
Statistics is a useful metric, but how they are used and what they are trying to say while using them can often be very subjective.
Example USA-someone is collecting unemployment, in December their unemployment money is up. They have no job, they want one and are looking.
By government standards in January they are no longer unemployed, they are "not part of the work force" and ding! guess what unemployment rates just dropped because that person is no longer unemployed, awesome!
Anyone interested in the reality of who is unemployed knows that statistic is full of bs. In December that person is looking and wants a job and is unemployed. In January that person is looking and wants a job and is unemployed. Stats changed but reality didnt.
Inflation statistics have the same problem. They typically don't include house, stock or bond prices. At best, they include some convoluted and highly questionable derivative of house prices like estimated rents. At worst houses don't affect the indexes at all.
This leads to a situation where central bankers are publicly saying "hmmm, we've done massive QE but there's no inflation, what a mystery" whilst the average punter is seeing housing and stock market bubbles inflate right in front of his eyes. It causes people to tune out: headlines like "inflation statistics hit new low" are placed right next to "house prices reach record highs".
It's not that the authority of statistics is lost on me. What I don't trust is the reporting of statistical information without transparent access to the data and actual published results from knowledgeable statisticians.
I think the solution is to push calculus off the K-12 agenda and up to the college/university level and replace it with probability and statistics instead. Most people understand averages, but they have no clue what standard deviation is. In fact (sad to say) many software developers don't even know what SD is!
Until the general public are able to understand the power of numbers and their pragmatic use, people who know how to manipulate the masses will continue to do so.
Yeah. The article makes a case that people have come to a newfound mistrust for statistics.
But, that mistrust has been around for a very long time.
A big part of the mistrust is not in the numbers themselves, but in the people who trot out the statistics and the way they carefully cherry pick numbers that support their cause, ignoring other numbers.
the way they carefully cherry pick numbers that support their cause, ignoring other numbers
Sometimes not so carefully. In a previous election, I remember seeing a purported screenshot from an evening news show that showed poll results, but skipped the #2 candidate entirely. Presumably they favored #1, and didn't want people to think there was a viable contender.
This screenshot itself could have been faked, but we've seen other serious blunders by TV news, like the fabricated names of officers on a downed flight.
A recent example I saw was a friend of mine on Facebook, who posted a snapshot from an MSNBC program. It was an attack on Rex Tillerson, in the form of a bar graph of profit potential for ExxonMobil by country.
The countries seemed pretty random (that well known oil giant Germany was in there) but at the top was Russia with a bar far far larger than any other country on the chart. The graph was an obvious attempt to continue the theme of "Trump administration is owned by Russia". The problem was that the numbers next to each bar weren't labelled: the graph was totally lacking in context.
I did some googling and found that the graph appeared to be measuring profit potential in millions of acres of drilling rights. Thus it was comparing an acre of frozen Arctic ocean with an acre of Germany as if they were the same thing. There are industry standard ways to compare oil fields. I've never once seen oil fields measured in this way because it ignores extraction costs, but sure, if you want to imply that Russia will be wildly more profitable than anywhere else on earth then that's one way to do it. But it's an abuse of statistics.
I pointed this out to the friend in question who simply said that she had a policy of not taking part in political debates on Facebook.
A problem I found with article is the talk of opacity of statistics techniques in the guise of "big data" used in social media corporations. Statistics used by individuals and companies is nothing new. As a counter-example, I feel the public has greatly benefited from statistics in the form of quality management embedded in the proprietary processes of manufacturing and large service corporations. Stuff like TQM allow us to take for granted so many things. I think the issues revolve around who owns and gets access to large data sets, mathematical literacy and the morality/ethics of various players.
Perhaps from a moral perspective, statistics about inanimate matter like TQM are better than statistics about people? It's somewhat reminiscent of the Second Formulation of the Categorical Imperative, anyway.
"...Is there a way out of this polarisation? Must we simply choose between a politics of facts and one of emotions, or is there another way of looking at this situation?..."
I feel extremely uncomfortable being put in the position of supporting populist, right-wing causes simply because the essayist is doing such a bad job, but here we are.
This author did a terrible job of understanding and explaining whatever the hell it was that they were trying to understand and explain. Perhaps "Populists hate science! How can we live with them?"
Heck with it. I'm not going to take the other side. There are a half-dozen problems with how the argument is set up. Anybody with a passing familiarization with "There are lies, damned lies, and statistics" should at least be able to set up a counter-argument or two.
Statistical science is great. It's the foundation of much of the western economies. "Statistics", as that kind of numerical smorgasbord that's thrown around constantly in the media and politics where correlation and causation are the same? Not so much. If you can't see the difference, you really shouldn't be writing essays supposing to inform us on anything.
The "statistics" is only in decline because it was blatantly used to try and sway the election. As in, oversampling demographics to get the necessary conclusion, phrasing the questions such that you're "literally Hitler" if you answer them "wrong" and so on. I quoted "statistics" because it's not statistics, it's pretty obvious propaganda, using the most shameless tricks in the book. Editors of Soviet Pravda would be proud. We know much of this thanks to Wikileaks now, as well as because it was so blindingly obvious that only a complete moron would think the polls were impartial.
I'm not afraid of "what's next" because as far as political polling is concerned (calling it "statistics" is way too generous), it just can't really get any worse than it already is.
Even the polls were cooked. E.g. to improve the odds of polls fitting the narrative, democratic leaning pollsters would disproportionately sample minorities known to vote predominantly democratic. The patron saint of pollsters, Nate Silver, was consistently wrong about everything all the way through the campaign. Now, if his main goal wasn't to peddle a particular narrative, one would expect him to make corrections to his models after getting things wrong, but no, he sacrificed his professional integrity instead.
I'm sure this is not the first time this happened, but this year it was particularly obvious, and I don't really trust the polls at all anymore except when it comes to completely non-political topics.
Nate Silver gave the eventual winner a 30% chance of victory, and his prediction for the popular vote was off by only 1% for Trump and 0.5% for Clinton. That's a damned good job if you ask me.
I can't fathom people who look at the results of this election and conclude that the polls are all horrible. They got very close to what happened. Many places took those poll results and interpreted them far beyond what they should have, saying Clinton was a lock when she wasn't, but that's a different matter. (It seems like they assumed that errors in swing state polls would be uncorrelated, thus vastly underestimating the chances that a lot of them would flip together.)
If you were trying to sway the election for Clinton, talking about how her victory is assured would be a bad way to go about it. That just encourages her voters to stay home. If the polls were biased that way, and if it was a deliberate attempt to sway the election, then if anything it would be an attempt to sway it for Trump. Which really doesn't seem likely at all.
How is giving a 30% chance of winning to the person who won "the wrong result"? Do you think that anything over 50% odds means a guaranteed win or something?
Silver gave Trump 2% chance of winning the primary. He also wheeled out this doozie a day before the election when it was pretty clear that Trump knew he would win. You don't do 5 rallies a day if you know you're losing.
For the polls to be cooked you would need the active participation of literally hundreds of different polling companies, coupled with none of the employees of those pollsters deciding to reveal the vast left-wing conspiracy.
I don't think most of those complaining about the polls have any idea just how hard polling is to get right, even if you're being completely honest. There is a shit load of educated guesswork in the turnout modeling and the demographic adjustments, and that the polls get as close add they do is a minor miracle. Go and read the British Polling Council postmortem of the 2015 General Election failings if you want to look at the gory details: it's really quite transparent.
Not to say there aren't careless and partisan pollsters in the world: there are lots of both. But there's no need for conspiracy theories when the reality looks like this.
I was about to point out that your use of "oversampling" didn't refer to the standard statistical technique, but rather to the right-wing conspiracy theory about polling being intentionally altered, but you've pretty much made that clear with your reply.
I think this first became a "thing" back in 2012, when the polls were predicting Obama to beat Romney, and that just didn't seem like it could be true, and it was blamed on "oversampling".
Also, Nate Silver isn't a pollster, he just uses their output as input to his models.
Campaigns often conduct private polling that they pay for, why should this be illegal? I don't see anything wrong in that email - all it means is that they wanted to get more resolution for certain demographics, I think.
No, they didn't want "more resolution" for "certain demographics". They wanted these demographics overrepresented in the final sample, without disclosing the fact they are overrepresented. These are polls for external consumption, the kinds you'd then find on CNN, HuffPo and MSNBC. I'm sure they also had their own private polls which, while probably also cooked, were more accurate.
The weird thing is that there are statistics that are based basically on surveys, whose collection and analysis is essentially model- and assumption-free. Like the unemployment rate, or inflation, for example. And yet there are enough politicians who reject even those statistics, and that seems like a totally different category of stupidity. When that happens, it's hard to believe that the fault lies with the statisticians.
Careful - both measures that you mention have baked-in assumptions. Inflation measures are based on a 'standard' basket of goods and services which needs to be chosen, and unemployment measures are dependent on definition: are all those looking for a job unemployed? All those on government assistance? All those working less than a given number of hours a week?
It was a favourite trick of British governments in the 80s and 90s to continually change the definition of unemployment, both to reduce the headline numbers and to prevent the construction of comparable data series.
One problem is that "unemployment rate" is a technical term that does not match the colloquial / folk understanding. "There is a 5% unemployment rate" and "25% of working-age adults do not have a job" are not mutually inconsistent statements. So if you "know" that more than 5% of people are not working but you keep hearing about the low unemployment rate (and multiply that by 10s or hundreds of other little things like that), it's not hard to make the jump to "the government is lying to us"
Your example seems more like an example of people not having a consistent mental model of what words means. A typical person would probably say that "a working-age person without a job is unemployed" and "a full-time student or a stay-at-home mom or someone who retired early are not unemployed" are both true statements, but the two statements are contradictory. A statistician cannot possibly come up with a definition of unemployment that matches the colloquial understanding of what unemployment means.
Of course they can. They can report the unemployment rate as "percentage of the population physically capable of working who do not", and then provide other terms to refer to "people who are looking for a job but can't find one", maybe call it the jobseekers rate or something.
Or more that the statistics trumpeted by policy makers do not match people personal observations. AKA Politicians and Policy makers and the federal reserve are pointing to a low U3 and slight wage increases to justify raising interest rates. And yet they totally ignore the employment to population rate, and ignore very long time it takes unemployed people to find another job.
I suspect that any purely empirical study is model dependent, even if the choice of model isn't obvious to the researcher.
My mom, a high school chemistry teacher, used to say, sarcastically: "If you want your data to fit a straight line, collect two points." And if you only collect two points, you've made a tacit assumption about your model.
In general, the form of your data set (what things you vary from one measurement to the next) determines the form of the model that you can fit to the data. This is particularly evident if the model function is a low order Taylor polynomial.
The interpretation of the results is another chance for hidden assumptions. For instance, referring to the inflation rate as "inflation" when it doesn't include the cost of higher education and health care, for instance, represents a bias that's easy to overlook.
Inflation is a great example of a statistic that appears to be model-free, but actually has some hidden assumptions behind it.
In my country, inflation has been comfortably in the 1-3% range since the '90s. But house prices have risen by about 6-8% per annum over the same period. So as a non-homeowner, the inflation I experience is substantially higher.
I think that's the problem with inflation. We should look at inflation for a given income level. If housing is 50% of a poor persons income and 2% a wealthy persons a 10% increase in housing costs seems to result in a diffrent decrease in spending power for each group. (5 and .2 respectively)
Surveys make gigantic assumptions, namely that the sampling is random or representative or can be made so.
How do you take a random sample of Americans? If you call landlines, you'll be heavily biased toward the demographic that has landlines. If you call cell phones too, you'll be biased toward the demographic that has phones. You'll also be biased toward people who answer calls from numbers they don't recognize, and are willing to spend ten minutes talking to a stranger about whatever.
You can survey people in the street, but then you're biased toward people who go out more, and again biased toward people willing to spend ten minutes talking to a stranger.
Obviously they know about these problems and try to compensate for them. And I'm sure they usually do a pretty good job of compensating. But that compensation involves a lot of modeling and assumptions.
Wait what? Public opinion surveys (if that's what you meant) are absolutely still subject to error. This is especially due to getting a representative mix of all of the different populations.
Even without those issues, you still have the issue of people you survey not telling the whole truth... or surveys being poorly designed so that the question people think they are answering is not what was really being asked.
Data is hard. Just because you have some survey data and (maybe) did something statistical doesn't at all mean you should listen to it.
I think it's generally accepted that unemployment and inflation estimates are based on reliable techniques: what sorts of errors are you thinking of here? I mean, in principle, anything you do could be wrong (you can never reach absolute certainty), but you would always do better by trying to analyse things properly than by giving up.
Edit: I didn't mean surveys in general, I meant the specific cases of unemployment and inflation, which to my mind are the two most prominent examples of people going crazy rejecting well-established facts.
The right tends to prefer to reference labor participation rate, whereas the left tends to prefer unemployment rate. The reason for this is obvious: right now, unemployment rate advances the narrative of the left, and labor participation rate advances the narrative of the right.
The argument from the right is that unemployment rate does not include the very large number of people who have given up searching for work (and have found some other way to survive, be it government assistance, or family, or charity, or illegal activities, etc.). This argues that the economy is actually in much worse shape than the unemployment rate would suggest.
The argument from the left is that labor participation rate does not clearly indicate who really needs or wants to be working, only who is and is not working. This argues that the people out of the work force don't actually need to be in the workforce and therefore the economy is actually fine.
If ever the unemployment rate works out better for the right, and labor participation rate better for the left, you will see the sides switch which metric they prefer to reference. That's how humans work.
Calling those who don't subscribe to your preferred narrative "stupid" and "crazy" is probably not helpful.
> This argues that the economy is actually in much worse shape than the unemployment rate would suggest.
This argument usually doesn't work because if you consider a different subset of the people who aren't working, you must then compare it to a different baseline. So being able to pick a different set of people to call unemployment (e.g., people not in labour force) doesn't actually let you claim the things are much better/worse, you still need to work further. In particular with respect to the labour participation rate, I think it's well known in leftish circles that it went down quite a lot in the GFC and that that represents a problem. I don't really know where you got your impression from.
In any case, I disagree that that's the argument they are usually making when questioning unemployment statistics. It could be their argument, but they'd have to actually make that argument. There usually isn't much beyond just rejecting statistics, unfortunately.
Furthermore, there is demonstrably a lot of people who think the inflation figures are downright made up (even here on HN), despite plenty of evidence to the contrary. In fact, the inflation example is even clearer, due to how little leeway there is in deciding, for example, what to call inflation.
In any case, I didn't call them "stupid" and "crazy" because they "didn't subscribe to my preferred narrative", as you put it. It definitely wouldn't be stupid or crazy to make the argument you are making. There has been plenty of discussion in economics history of which measures best represent the state of the economy. However, if someone, as many politicians do, feels comfortable enough to reject that because they don't feel it's convenient, then, sure, "stupid" and "crazy" are decent enough words to describe that particular behaviour.
After all, just because I can come up with a better argument for someone's position, doesn't mean I can substitute the argument they made with a different argument that I find more reasonable (it would be distinctly uncivil). Specifically with respect to unemployment rates, you and I may simply have been reading different things, but I believe what I said was accurate.
One further thought: the unemployment rate (not the participation rate) is part of the Federal Reserve's mandate, and it is the rate that has traditionally been used, with perfectly decent theoretical justification, so I feel quite comfortable saying that it's not some kind of left-right partisan divide.
> Specifically with respect to unemployment rates, you and I may simply have been reading different things
Certainly possible, but what I generally see is:
"the unemployment rate is not what [democratic administration] says it is because they redefined the metric to not include those that have given up trying to find work...the labor participation rate has dropped several percent since the beginning of [democratic administration], and the unemployment rate has also dropped several percent in that time, and these cannot both be the case."
Often the sound bytes we're shown only show the "I don't believe it" part, but if you dig a bit, at least I, generally find more context and clarification in other statements from the individual (often in an expanded version of the same quote in the sound byte), showing that the above quote is roughly the thought process of most challenging the unemployment rate statistic.
What the sides are really arguing about, as is the case with most "statistics" used in politics, is what the definition of "unemployed" should be.
Re people dismissing clear facts, again, this is just human nature. The brain finds a way to twist reality around facts that challenge foundational beliefs. The more foundational the belief, meaning the more structure built on top of it and the more decisions you have made based on the belief, the harder your brain will work to twist reality to allow you to dismiss this inconvenient fact. All humans do this, and both political wings in this country do it equally. Somewhat meta, but believing that only the "other side" is victim to this behavior is yet another example of the behavior (and we all do it).
So first of all, any significant drop in employment is overwhelmingly likely due to the GFC.
Second, I must take issue with the idea that they (the administration) "redefined" the metric. As I said above, there is a genuine (and basically settled, I believe) question in economics about what the right way is to summarize the state of the economy's labour force in a number. So any concern with it is going to have to be a technical concern: does a measure summarize the state better or worse than another measure. In fact, the BLS consists of professionals with a specific job to do, so I would be really skeptical of a claim that the administration redefined a metric that was inconvenient to it. It is rather more likely that it was already settled on economics grounds previously.
It would be really surprising if the BLS, an independent statistical agency, did something like what the argument accuses it of doing, so there needs to be some more solid evidence that it actually happened.
Specifically with respect to people who gave up looking for work being included as unemployed, I'm pretty sure that's been like that for ages now.
Third, and my main problem with that argument, is that it's an argument ad hominem. Instead of settling the choice of unemployment measure on merit (economics is a relatively old discipline, there is more than enough to talk about to decide what the merits are), the argument instead tries to argue based on imputed motivation of the current administration. That's a very dodgy road to go down, because settling it on merit is always a better thing to do.
> these cannot both be the case
It's not my main field of expertise, but this, I believe, is just plain wrong. The structure of the economy can definitely change with time. Two prominent examples to my mind are: (1) women entering the labour force, leaving the unemployment rate unchanged, but drastically increasing the number of working adults, (2) changes to disability and health insurance rules that sometimes mean that people who are seriously ill drop in/out of the labour force with good reason (they might be very sick yet still need to work to be able to afford things). So I think it definitely can be the case that different measures will go different ways, depending on the underlying changes in the economy.
> Re people dismissing clear facts, again, this is just human nature.
Meh, I don't care, I just want to get it right.
Edit: On further thought, I think the argument you mentioned is pretty damn close to dismissing statistics. Your original argument, above, was totally different, and much better.
I think you misunderstood much of my last comment, sorry if it was my fault.
First, none of these are really "my argument", I was just explaining some of the behavior that you are witnessing and didn't seem to understand (the reasoning behind some on the right dismissing the unemployment statistic as well as humans in general dismissing data that causes problems with their foundational beliefs), though I do tend to lean more towards the right's side on this issue.
As to the argument of those who have given up looking for work not being included: whether or not those who have given up looking for work have always been excluded from the unemployment measure is not the point of the argument. What they are saying is "8 years ago (or whenever) when the unemployment rate was (say) 7.9%, the count included a large number of people who have since stopped looking for work..these people now no longer count as unemployed today when the rate is 4.9%, and very significant part of that drop is from people just giving up, and not from an increase in people finding work". You could argue that over any given range new people are entering the unemployed count and people are giving up, and that over a long enough period this behavior is not statistically significant, but the counter-argument is that during this particular period there were massively more people just giving up than at any other time, and so the drop in the rate isn't actually a reduction in the count of unemployed.
As to both the labor participation rate and the unemployment dropping a significant amount during the same period, of course there are conditions that could lead to this, but the point is that it is not intuitive and the causes for it really need to be explained for either of these number to have any real meaning.
Nobody (mostly anyways) is arguing that 4.9% of respondents to a survey didn't actually indicate that they are unemployed. What people are arguing is that the definition of "unemployed" being used doesn't accurately reflect the "real" unemployment rate.
One further thought: I was looking at https://fred.stlouisfed.org/release/tables?rid=50&eid=4786, and it seems that both the unemployment rate, and the U-{1,2,3,4,5,6} measures are approximately at their historical values right now. So it seems that when the unemployment rate shows unemployment at normal historical levels (5% is about the equilibrium rate), the other underemployment rates show the same thing, which makes using the unemployment rate okay. So I think there is quite some danger of looking at the data, seeing what it shows, and not believing it (thus "dismissing" it).
I don't think I misunderstood your comment. I think in decisions about who to include in unemployment statistics, one should just defer to the professional body in charge of them, unless there is a clear methodological reason not to. There is no particular reason to think any rate is the "real" rate, so while it is true there are multiple rates you could compute, the right thing to do is to pick the standard one. And so anyone who relies on the standard rate (politicians, the Fed) is quite correct to do so. Besides, I'm not sure about this, but I have the feeling the unemployment rate by design isn't meant to track structural changes in the economy (and that's fine!), so if you find a structural change in the economy it doesn't track, you'd still be okay using it.
> Is there only one official definition of unemployment?
There is only one official definition of unemployment—people who are jobless, actively seeking work, and available to take a job, as discussed above. The official unemployment rate for the nation is the number of unemployed as a percentage of the labor force (the sum of the employed and unemployed).
> Some have argued, however, that these unemployment measures are too restricted, and that they do not adequately capture the breadth of labor market problems. For this reason, economists at BLS developed a set of alternative measures of labor underutilization. These measures, expressed as percentages, are published every month in The Employment Situation news release. They range from a very limited measure that includes only those who have been unemployed for 15 weeks or more to a very broad one that includes total unemployed, all people marginally attached to the labor force, and all individuals employed part time for economic reasons. More information about the alternative measures is available on the BLS website.
Ok, it's clear you are just trolling at this point. It's really not that difficult. In October of 2009 the unemployment rate was 10.0%, and today it is 4.9%.
Democrats use this stat to say "look how many people the administration has put back to work". Republicans then say "easy there, most of this 'improvement' is from people just giving up, not from all of these people finding work".
Nobody is arguing that the stat is being _recorded_ incorrectly. They are arguing that it is being _used_ incorrectly. It is being used to imply that there has been a massive improvement in the availability of work. The other side says this stat does not actually show an improvement in the availability of work. What is meant by "real unemployment rate" is a stat that would better show the availability of work.
If you don't like that name, then name it whatever you want, but don't pretend that you don't know what people are talking about when they use the term. That is the "dangerous" behavior as far as I am concerned.
I think you are wrong to bring politics into this, especially the left-right divide. The question of whether the unemployment rate is meaningful and useful is a question of economics, and should be settled as such. In fact, it is being used exactly as it is meant to, consistent with its historical use. Like I said in the other comment, since the other measures of underemployment show similar results, that makes it that much more likely that the unemployment rate is reliable.
Also, please don't throw trolling accusations like that.
You started this thread with "And yet there are enough politicians who reject even those statistics, and that seems like a totally different category of stupidity".
Now you say "I think you are wrong to bring politics into this".
This entire subthread was about politicians' use of this statistic. I've mentioned several times now that the disagreement over the statistic is a disagreement over its use by politicians, not over the accuracy of recording the statistic, and you continue to sidestep this. If that's not trolling, I don't know what is.
The average person has one testicle, one ovary, and less than two legs. I'm being facetious, but my point is that highly aggregated data can lose nuance.
Personally I mistrust the inflation numbers that the government publishes, since I think there are major intergenerational effects that aren't captured by the figure. Young people have to contend with enormous student loan bills and massively inflated house prices, while these aren't issues for older people.
In short, I don't think it's appropriate to try to describe the evolution of the purchasing power of an entire population using a single monthly figure.
You are one of several people whose take on this seems to be "no one trusts statistics, we can fix this by introducing new, more complicated statistics, that better fit my personal argument"
Now, I'm personally interested in new ways to look at the data, but in a world where everything is "rigged" what you're proposing sounds even more "rigged".
I didn't actually propose that any new statistics be published, I just said the current ones are lacking.
To say that "It's not appropriate to use a single monthly figure" is not the same as saying "We should publish a number of figures broken down by age, location, and other factors". I honestly don't know what the right approach is.
The adjustments that are included in the official unemployment rate in Australia are decided on by a committee. Recently, Oct 2014, they decided to change them and got it all wrong, giving a very different rate. http://morningmail.org/can-count/
You could do a survey about peoples' eating habits, and no matter how you go about it, the results will be fraught with error, despite it being arguably objective.
Most people have a pretty poor idea of just what and how much they are actually eating.
There's a gazillion assumptions baked into inflation measures. The basket of goods to track, accounting for changes in consumption patterns, accounting for technological improvements, etc. There is no objectively correct answer to any of those issues.
Statistics never changes, what changes is the people who interprets them. So if there is anything to blame, elites should blame themselves because people don't want to believe their interpretation anymore.
Reminder: the terms "statistics" and "statisticians" come from the common root word "statist".
I am a trained, practicing statistician (clinical and medical, not governmental) and this always gives me pause. "Data science" is a shitty term, because science without data isn't science at all. Moreover, these days it's just long for "IT guy". But I'm not sure what's better as a term for the real thing, where experimental design and inference are as important as predictive accuracy in one specific context.
Anyways. Statistics was always built around statism.
This is kind of a nonsequitur. The etymology of a word does (obviously) not imply anything about the social impact that the use of statistics has and is just a clever anecdote. I haven't seen any marxist criticism of statistics itself, which is telling considering marxism is a ruthless critique of everything. Obviously a state or corporation can use any tool (science, technology, writing, math) for its own gain or detriment, and that fact speaks nothing to the general social utility of that tool- it also doesn't speak to whether or not specific uses of that tool by a state or corporation are legitimate or self-serving.
Also, "Data Science" doesn't refer to Science with Data, but the study of data analysis itself, and for that purpose is a reasonable albeit often unnecessary term.
He's wrong, that's not correct. In fact it comes from "state", statism is a term referring to the actions of a highly centralised government who handle economic planning and controls.
"Statism" is a term that has been around since the 1850s, but "statistics" is a term coined by Gottfried Aschenwall in the late 1770s.
Well, I didn't claim to be a trained historian, which is probably the correct training for this type of work. Based on your temporal evidence, I must concede the point.
https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/03...
Statistics is ultimately counting, and therefore is incredibly vulnerable to discretion in choosing "what counts". Take the unemployment rate as one example. When people realize that its not equivalent to "people who do not have a job", how can you complain that they trust statistics less?
Expert authority is in decline because it should be, as there is an increasing body of evidence that experts, from politics to medicine, have almost no advantage in forecasting power than the average person.
https://www.amazon.com/Expert-Political-Judgment-Good-Know/d...
Why should "experts" (often just pundits) have any authority when they have consistently demonstrated they deserve very little?
Finally, the political slant of this article, going along with the decried "fake news", blaming the election results on these declines in authority, is pathetic. It's basically an extension of "the other side is filled with stupids" and has no credibility, no matter how you dress it in professional journalistic veneer.