While the article obliquely makes reference to this, I wonder if this is somewhat similar to the "birthday" problem in statistics. Any given person has ~1 in 365 chance of being born on a particular day. Yet, if you have 30 people in a room, there is a greater than 50 percent chance that at least one PAIR has the same birthday. Given the hundreds of cities (thousands?) in the US that may be affected by a flood, it's actually quite likely that at least one of them could have a pretty high probability of experiencing one in 500 chance of something each year. I haven't run the numbers, of course, but we're looking at Houston in isolation - it needs to be seen in the broader context of all possible cities. We're focusing on the individual probability of Houston having this improbable string of events, rather than looking at Houston only being the city that experienced this phenomenon.
Of course, the "once in 500 years flood" may also be an inaccurate probability as well, but if its not, then I'm not surprised this is happening SOMEWHERE.
It is of course possible, it was actually a once in 500 year event. I just believe it is much more likely our previous conclusion was faulty.
Our prediction of 500 year floods is not very good, we rely on way less than 500 years of data. Also in most places where this happens the massive changes to the environment (roads, cities, paved over wetlands, constrained rivers...) are not factored in well at all. Add to that global climate change and 500 year flood estimates are likely still poor today.
The birthday problem doesn't relate to us getting new data that changes what we used to know. A 500 year flood probably is new data (that gives us a strong indication our previous belief was wrong).
This is part of the reason few private insurers offer flood insurance. They understand they there isn't enough information to properly price the risk. The historic data is spotty, we don't understand the geography well enough, you have to understand how construction and urbanization affects water absorption over long distances. If everyone who claimed to know what a 500-year storm looked like had to pay out every time one happened, we might have better models.
No, that isn't why. I worked in insurance. Not homeowners insurance, but I had a certificate and processed claims.
The reason is that most human settlements occur in flood plains because we cannot live without water. We drink it. We bath in it. We irrigate crops with it. Flood plains have fertile soil. We use rivers and oceans for essential cargo transit.
Insurance is about risk management. It is a form of betting. And there is no bet here because there is no question of if it will flood. The question is only when will it flood?
That's a fool's bet to say "I will pay you X amount of money if it floods" when it is guaranteed to flood sooner or later. That amounts to charity, not insurance.
Hey, I work in insurance too - as a non-life actuary for the last almost 8-9 years. Your description is not at all how insurance works, and insurance is not a form of betting or gambling but really distinctly different. What it's really about is just risk pooling; my share of the risk pool is smaller than if I would keep the risk to myself.
Insurance companies don't mind insuring risks that they know will happen at some point, e.g. most property policies will have some claims, and life term contracts (as have been stated below) of course will have a claim at some point (unless the policy is lapsed). However, they have an idea of how often and how costly these claims will be, and through risk pooling diversification, this is lower (per policy) than the cost to the individual. Thus there is an incentive to buy the policy, and an incentive to sell it - as the difference can be made as profit to the insurer.
In addition there's a timing element to insurance: insurers take in premiums "now" for claims that will be paid out "later", so they can invest the money in the meantime. Large insurance companies may have $200bn investment portfolios.
EDIT: So the point is that flood insurance can of course be sold by private companies, however they know the risk is too high and won't offer competitive premiums. If we only had private flood insurance in the US, in the long term this would lead to people having to move to places with cheaper flood insurance. In this way it actively promotes people moving away from risky places (which I think is a good thing) - but it makes it difficult for people in the short term.
You're both right. Cost pooling and betting are two ways of thinking of the same thing. If anything, betting is closer to marginal thinking than cost pooling, but the two are not exclusive viewpoints.
A lot of life insurance is term insurance. So the bet is "will you die during the years that your policy is in force?" Whole life is much, much more expensive.
Could you then have the same kind of structures for flood insurance policies? I had to pay every year to renew my renter's insurance policy, it only covered losses during the course of that year, and the insurance carrier had discretion to adjust the premiums or not to renew it.
In the US, flood insurance is provided by the government. And the premiums still jump insanely sometimes.
It gets handled this way for the same reason our government provides welfare et al: it makes no sense as a business, but the cost to the nation to do nothing is a bigger problem.
There is some private flood insurance these days, but I don't know who offers it, in what locations, what the premiums are, and what kinds of exclusions there are. Since it's a new thing, I think the claim that it'll be vastly better than NFIP is invisible hand waving. Nevertheless...
http://www.insurancejournal.com/magazines/features/2017/07/1...
I think the bigger issue is that there is an inherent incompatibility between free markets (or strict laissez faire, non-intervention of any kind) and democracy. Of course people, in the millions, are going to say "help us" and direct it at their various layers of government, and punish those who don't at the election booth.
Therefore it stands to reason in major floods like this, that everyone is going to get some kind of relief even if they didn't have flood insurance. What I'm not sure of is whether the insured get 100% payouts and those not insured get partial payouts? What's the incentive to have flood insurance, except in smaller, localized, 50 or 100 year floods?
This is asset destruction and the only way to properly handle it is through savings. So it's either made compulsory or you do end up with something of a free loader problem. Whether that free loader problem is a real problem, I don't know.
How could term insurance for homes be possible? Homes are (despite some fluctuations) relatively good stores of value that you want to pass on to later generations. Comparatively, once a typical human retires from working, he isn't producing much of insurable value (and presumably, his offspring are capable of achieving financial independence).
Homeowners insurance is all term insurance, the contract only covers a short period of time. At the end of the contract, the policy holder doesn't have anything.
That's not the problem. Private insurance does not offer flood insurance because A) it tends to all be due at the same time requiring a very large fund. B) It also promotes people living in flood prone areas. C) It's very hard to asses risks. Combined and you end up with bankrupt insurance companies.
The industry also feels B makes selling flood insurance a bad idea. That said, there are Private policies you can buy in some areas.
PS: Insurance companies like frequent small scale random events like car accidents, because they are easy to plan for.
I don't think point B is valid. Following your reasoning, the industry would also think that health insurances are a bad idea because they would promote an unhealthy livestyle.
The company would have to be huge to pull it off. One event like what happened in Houston could easily wipe out a big chunk of any fund they could amass. That kind of diversity could also be a problem if there are shared waterways, think back to the Mississippi/Missouri flood from 1993. I'm seeing 15 billion as the damages there, which is a large percentage of the "cash" pile Apple is claimed to have.
I believe it's a reference to the fact that even large companies, Apple is $800b, don't have all that cash on hand. To convert $15b into cash takes time. Disaster victims can't exactly pay for hotels or buy foods with bonds or stock certificates.
That's a fool's bet to say "I will pay you X amount of money if it floods" when it is guaranteed to flood sooner or later. That amounts to charity, not insurance.
A statement with profound implications in other areas of insurance, notably health insurance. You're insuring against expensive losses that will almost certainly happen at some point... which suggests that traditional insurance isn't the right framework for solving the problem at hand.
Yes, but health insurance worked differently before Obamacare. Insurance companies simply did not cover chronically ill people or people with preexisting conditions, thus they were able to arrange a "bet." Obamacare removes the element of a "bet" from the system.
Just like the government provides fire protection in the form of fire departments, the government needs to provide health care. What we are doing currently is broken.
Structuring healthcare as an insurance doesn't work because pretty much everyone needs it at some point. It needs to be structured like a tax, because the rate at which you pay in has no correspondance with how much you need to get back out, but in aggregate the total sum paid matches the total care provided. That's why the successful healthcare systems in the world are typically single payer systems. However, such systems are politically untenable in the U.S. and so you get obamacare. I'm sure if it was within the overton window of acceptable political thought obamacare would have been a single payer system.
> Structuring healthcare as an insurance doesn't work because pretty much everyone needs it at some point. It needs to be structured like a tax, because the rate at which you pay in has no correspondance with how much you need to get back out, but in aggregate the total sum paid matches the total care provided.
you've just outlined a money-losing system run by the government.
the reason private health insurance systems fail is not merely because everyone needs it at some point. it's because everyone uses much more of it than they've paid for with their premiums. (which is why, pre-Obamacare, insurance companies used to reject applicants with pre-existing conditions and why they put lifetime caps on benefits paid.)
but a government run single-payer system is not magic. the extra cash must come from somewhere.
one way a single-payer system could obtain the additional funds needed is by taking cash from some other government source (which is exactly what Obamacare does when it provides refundable tax credits to help low income people pay their otherwise unaffordable health insurance premiums.) but there's no end to the amount of money that could require.
another thing the government can do is limit services to patients, i.e. take a certain amount of decision making authority, by law, away from individuals, especially old individuals who are very sick. ("i'm sorry, but we're not paying for that new chemotherapy. it's hospice care for you.")
in addition, a single-payer system can also uniformly limit the prices health care providers, pharma companies, hospitals, etc can charge. (i.e take some decision making authority away from that sector of the economy, again, by law.)
I live in a single payer system country and it's nothing like what you describe.
First, how it is paid. It's text book socialism. People pay into it based on ability and receive benefits based on need. It doesn't run at a loss.
Second, the level of care. Basically the government puts a lower bound on health care and uses taxation to do it. You're guaranteed a basic level of care regardless of your situation. You can opt for private insurance that stacks on top of that. A basic level of treatment is covered by the government, but anything experimental or exorbitant you will likely have to pay yourself, unless you have the aforementioned gold-plated insurance. Insurance is much cheaper because it only needs to cover unusual care, but of course I do pay a lot of taxes.
It's not different from a private system, because you can't receive unlimited care, but unlike a private system you're always guaranteed a minimum care.
If I'm not mistaken, they also had yearly or life-time limits, removing the "losing" part of the betting from the system.
Insurances are not bets, not more than other commercial undertakings. They are, for the insured, hedges against bets. The bet is: I will not get sick with something that will ruin me financially. Or: My house will not burn down.
You pay a steady, comparably small (to the risk) amount of money. Still the total amount of money paid by everybody is bigger than the total amount of pay-outs.
In the case of these massive flooding, it's hard to make the pool big enough. That's why reinsurance companies exist, which can mix the risk pool of different types and locals.
The patrons bet, for example, on whether Admiral John Byng would be shot for his incompetence in a naval battle with the French. He was.
The gentlemen of Lloyd's would have had no qualms about taking my bet on my own life.
Edward Lloyd realised his customers were as thirsty for information to fuel their bets as they were for coffee, and began to assemble a network of informants and a newsletter full of information about foreign ports, tides, and the comings and goings of ships.
His newsletter became known as Lloyd's List.
Lloyd's coffee house hosted ship auctions, and gatherings of sea captains who would share stories.
If someone wished to insure a ship, that could be done too: a contract would be drawn up, and the insurer would sign his name underneath - hence the term "underwriter". It became hard to say quite where coffee-house gambling ended and formal insurance began.
Usually, both. Plus, if you had high expenses they'd actively search for an (even unrelated) pre-existing condition as a pretext to cancel your coverage (recission).
Obamacare is not a terrible idea, it's a transitional idea as we're already doing this with insurance; it's an evolution of the system in the right direction, it's not a final solution. Add a national exchange, offer a public option, bam medicade for all.
That is called single payer, government provided coverage. I would support that.
That is not remotely what Obamacare does. Obamacare requires private insurance to cover people with pre-existing conditions and it requires everyone to buy private insurance. It is busted as all fuck and I would like to see it go die in a fire.
I am aware what we were doing before Obamacare sucks and we need a real solution and to not simply go back to that. But this is not a real solution.
Of course it's not what Obamacare does, you can't go from a private insurance market to a single payer government system in one single step. Obamacare is the transition between the two.
> Obamacare requires private insurance to cover people with pre-existing conditions and it requires everyone to buy private insurance.
So does single payer, we just call "buying" taxes and we replace insurers with government. You're still forced to pay and the insurer is still forced to provide for everyone regardless of pre-existing conditions. So it would seem you're simply against the name Obamacare.
> But this is not a real solution.
It's not supposed to be, everyone isn't blind, they see that single payer is the solution, but you can't simply declare the private insurance industry we already have dead in one fell swoop. Obamacare or something just like Obamacare is a necessary step to get to single payer. We need to get everyone into a public option, and then have that option slowly kill off private insurance by simply operating cheaper than they can until the public option is pretty much what is insuring everyone, at that point, it's effectively single payer.
> That's a fool's bet to say "I will pay you X amount of money if it floods" when it is guaranteed to flood sooner or later. That amounts to charity, not insurance.
The bet is more like, "I will pay you X amount of money if your damages from flooding exceed your deductible within the next year." It's not at all given that it will flood at all this year, or that your losses will exceed the deductible.
It's true that it would require substantial diversification or re-insurance, since flooding is a clustering event.
You wouldn’t be provided with a fixed lump sum, but rather your costs would be covered up to a limit.
So it’s about determining the expected value of losses for a given risk. You’re right that given a long enough time, there is a near-certain chance of a flood, but the question is more about how much that’s likely to cost.
Major earthquakes are less frequent than major flooding. If the time frame is large enough and the timing uncertain enough, it can still make sense as a bet.
Frequency only changes premiums, car insurance is much more likely to pay out than earthquake insurance per year. That just means it costs more money per year relative to the payout.
Well, insurers will often cover you for water damage from burst pipes, but not from flooding. That's because the damage from burst pipes happens at a predictable trickle of people over the course of a year, but damage from flooding happens to tens of thousands of people on the same day. So the amount of money you have to set aside for the two cases is wildly different. Side note: This is also why your life insurance policy is useless in the case of nuclear war :)
I don't understand this. Even if they can't assess the risk in any one property, can't they estimate the total amount of flood damage in (say) the US each year, and spread the risk across thousands or millions of properties?
If the answer to that is that only people in areas vulnerable to flooding would buy insurance, and the companies would have to pay out too much - well, that means the insurance buyers are pricing the risks better, and the insurance company should hire some of them.
To re-emphasise what the poster before you said - not only could the model be wrong, but even if it was right once before, you have to keep up the model against all the construction changes in the area that alter the watershed, as well as account for climate change contributions.
Their model is wrong, but it's because it assumes climate is stationary. Climatologists are actively debating what the models should look like when global warming is taken into account:
One of the things that is expected with global warming is heavier rain as the seas get warmer. This 500 year rain calculation is looking at the past for prediction but the future is different.
Does 1 in 500 come from assuming a normal distribution? Seems you would also need to accommodate random events like "tropical storm parks itself off the coast" which is more like a poisson process.
The other problem is that these definitions are time independent - they treat the probability of each event as a roll of die, when in fact each previous event impacts the probability of future events, by impacting surface features, moving centres of production, driving further energy expenditure and altering haloclines and other such variables.
Your comment goes on to show that human intution aligns with an assumption that things are distributed with a thin tail (for example Gaussian like). On the other hand if the phenomena has a fat tail (Pareto for example) then runs of such 'extremes' are not unusual.
What is interesting is that one of the motivations (funding wise) for understanding fat tailed distribution, extreme value distributions were indeed flooding and dyke failure events.
> It is of course possible, it was actually a once in 500 year event. I just believe it is much more likely our previous conclusion was faulty.
It seems it could go either way to me, and the way to get a good indication of which is most likely is to look at look at much smaller timeframe models and assess if they are changing over time, and how much. It's entirely possible it's both, but it seems we have a good way of gauging the relative importance of changing climate norms so we should at look at that before discounting it as noise in the bigger problem of poor models in general.
Good point, but the chance of a given city having a 500-year flood in three given successive years is only (0.002)³ = 8 × 10⁻⁹. So even if we allowed for 100 U.S. cities, 10 different natural disasters and 10 different three-year periods we're still talking about a probability of less than 0.01%.
...the assumption being that the risks are separate. This is clearly false - you are not rolling dice here, the probabilities of the same disaster at geographically "close" cities are linked.
That is true but it's an error in the unproblematic direction: fixing it makes the argument even stronger.
That is, if the probability for each city flooding is 0.002 but the different cities are positively correlated, then the probability of at least one city flooding is lower than it would be if they were uncorrelated. (Because the probability of no cities flooding is higher, because of positive correlation.)
Having lived in Houston, this isn't a measure of the flood but a measure of how terrible the infrastructure is. They have terrible flood management systems but are "rated" to handle everything but a "100" year flood... yet Houston gets 100 year floods every year- multiple ones- and has for 40 years.
It's a scam involving the ratings much like the mooody's ratings scam of the last decade IMNSHO
I understand your argument, but you can't compare floods to the birthday problem. Unlike randomly-sampled people's birthdays, floods in US cities are not independent random variables.
For instance, chances of 500-year floods in two Texas cities is not 1 out of 500*500 in a year. If they're geographically similar and physically close, it should be closer to 1 out of 500.
In 2 Texas cities, sure, but in every U.S. city? There are going to be several that end up getting a few 500-year floods... and several others that don't flood as expected in several 1000 years.
>> Of course, the "once in 500 years flood" may also be an inaccurate probability as well
I suspect that's it. Of course I may be falling into the same birthday paradox here, but a similar thing happened when it flooded where I lived now. The x in "x-year Flood" kept increasing (I'm inclinced to blame that on sensationalizing news for ratings), but people kept comparing it to the 3 or 4 similar floods that happened in the last hundred years. It would seem the probably has gone up due to climate change, or that we're noticing the floods more now because there's more damage to be caused to more things that have been built up, and we have better records and better news transmission, but it certainly seems that can objectively say they're no longer 1-in-500 year flood regardless of the reason.
>. It would seem the probably has gone up due to climate change
That probably has a small reason to do with the increase, but your next line highlights the issue.
> or that we're noticing the floods more now because there's more damage to be caused to more things that have been built up
No, not exactly. The more stuff you build, the faster water runs off. Unincorporated land has a pretty high water slowing capability. It takes more time for water to run through tall grass, and puddles, and log jams, and loose dirt. All things that occur naturally. Humans build houses that have all the water run into gutters, that run into a well cut lawn and down a sidewalk. Once it hits the road it's like a water expressway. It can travel quickly to a local stream. A stream that likely has a concrete lining so the water moves swiftly to a lake or reservoir in hours rather than days.
As a place develops new flood plans must be studied every 5 to 10 years. A new development of 100 hours up the watershed from you can mean your risk of flooding increases dramatically. Places like Houston have had 10,000's of houses build in the surrounding areas meaning what was a 1 in 100 year flood could be as low as a 1 in 10 year flood. All this stuff being built is not taking into account its effect on downstream rainfall runoff.
I actually hadn't thought about that aspect at all. I was thinking more along the lines of 'more development -> more stuff to get destroyed', but that's kinda backwards anyway because a flood would probably wipe out a bigger portion of infrastructure hundreds of years ago than it does now.
But faster run-off sounds like a good thing to prevent the flood - are you saying that more development upstream means more flooding downstream because all the run-off arrives with much lower latency and higher bandwidth, to use IT terms?
Also people tend to drain marshes and any other shallow stagnant waterways to avoid mosquitoes having breeding places. It can save us from terrible diseases, but it can make flooding worse.
When Rainfall * Area >= SoilAbsorption(See note) + WetlandAbsorption + ArtificialDrainage the end result is flooding.
Essentially the left hand side of the inequality has been slightly increasing because of climate change, but we're chipping away constantly at the right side of the equation in three ways.
(1) More and more soils in American metro areas have been covered by impervious surfaces for development.
(2) More wetlands are being destroyed for development.
(3) Artificial drainage is being under-built.
Drainage is generally built for 1 in 100 year events but the rate of urbanization is increasing the magnitude of flooding. (Drainage is only a problem for the developer that hasn't sold what they're building.)
Note: Soil has a finite capacity for water infiltration.
There is one additional thing to note. Soil has a certain sponge like quality to absorb water. So a runoff curve number was created to "predict runoff or infiltration"
The tables on the wiki pages shows "curve numbers" for different types of soils. These numbers are based on experimental data which is then used to model runoff. (Or used to be. Its been 20 years since I've done runoff modeling, so maybe things have changed)
Up to a point. To continue the bandwidth analogy, having a buffer is cheap - unless the input link fills it faster than the output link can handle, for a prolonged period. The buffer gives you some extra time for processing, but when it fills, you are back to step 1.
There is always considerable political pressure to low-ball the estimates. That is hinted at in this article, and it happened in New York, where the first post-Sandy flood zone maps provoked widespread opposition from those in the newly-expanded zones, and they were rapidly scaled back. There is, of course, some uncertainty in the estimation, but it seems likely that the official maps will be, at best, based on the most favorable possible interpretation of the data.
Mother nature does not respect grandfather clauses.
There's gonna be lots of screaming when homeowners in Houston realize how much flood insurance is going to cost them in the future. We all got a preview of this in coastal Florida.
Pretty much everyone knows it's a good idea to get flood insurance in Houston.. We still get lots of localized downpours, floodplane or no, which can easily overwhelm local drainage, etc. Whether they actually do it or not is another idea
I am aware that flood insurance itself is subject to government influence - e.g. the National Flood Insurance Program, and possibly state regulation (even in Texas) - but I don't know the details. Does it result in premiums being affected by official flood-zone designations?
As you appear to be in or near Houston, I hope things are OK with you and your family.
From what I understand of the article, the "1 in 500 years" measure is based on the estimated probability of a flood on a given year in a local area. So the probability of a "1 in 500 year" flood in Houston is independent of the probability of a "1 in 500 year" flood for another city.
Right what the parent comment is saying is that look at a map of the US, how many major cities are there. The odds of two "1 in 500 year" floods happening consecutively are low for any individual city. But if you have enough cities it will happen somewhere. And then you write the article about it.
It's kind of analogous to hitting a golf ball into an open field and then exclaiming "Of all the blades of grass, the ball landed on this one!" It's only interesting if it goes in the hole with the flag, otherwise you have a sort of selection bias going.
That makes sense, I was confused since the Birthday problem is more about the pigeonhole principle rather than larger samples being more likely to have individually unlikely outcomes.
But for this case, it would still be very unlikely. For any arbitrary 3-year span, the probability of consecutively getting a "1 in 500 year" flood for 3 years is (1/500)^3. If we have N cities, then the probability of none of these cities having 3 consecutive floods is (1-(1/500)^3)^N. For even a vast overestimate such as N=20000, it is still a significantly improbable event. Of course, this doesn't account for "3 consecutive years or more within some year range" and it is a gross simplification, but I think there are probably better explanations than selection bias, such as the inaccuracy of the model or the fact that these events might be temporally dependent on each other.
My understanding is that by definition, a 500 year flood has a probability of 1/500 per year. But I live at the top of a hill in the Midwest, so a 500 year flood in my locale has a much different severity than a 500 year flood in Houston.
A more accurate term for what you describe would be a clustering illusion[1], and it certainly could be a valid explanation (among other possibilities) for the observation made in the article.
Birthday paradox relies on non-independent events. I'd assume floods should be relatively independent between different cities, meaning that the number of 500-year flood events in the U.S. would follow a binomial distribution (https://en.wikipedia.org/wiki/Binomial_distribution).
p = 1/500 years = 0.002
For 100 cities, the probability of at least one city having a 500-year flood is 18.14%.
For a 3-year run:
p = (1/500)^3 = 8e-9
For 100 cities, the probability is 8e-7 of at least one success. For 1000 cities, the probability is still only 8e-6.
This is actually a really interesting problem because depending on how you set it up, you'll get very different answers. To illustrate, notice the difference between these questions:
1) Suppose I flip a coin 3 times and repeat that experiment on 10 different occasions. What's the probability that I get all heads at least once?
2) Suppose I flip a coin 30 times (same total number of flips). What's the probability that I get at least 1 string of at least 3 heads?
(1) is more like the binomial calculation you've set up, but I think (2) is closer to the question we want to be asking.
This is important because, as it turns out, (2) is much more likely than (1). You can tell intuitively that there are more sequences of coin flips that would satisfy (2). For example "THH HTT ..." would satisfy (2) but not (1).
So how should we pose the hurricane question? I propose: What's the probability that in a 100 year period, across 100 cities, there will be at least 1 city that experiences a 500-year flood for at least 3 contiguous years?
Solving this analytically is non-trivial, but it's easy to simulate:
// Scala
val rand = scala.util.Random
def coinFlip(p: Double): Boolean = rand.nextDouble <= p
// Simulates 100 years and returns true if there's a contiguous string
// of three events with p=1/500.
val targetEvent = Seq(true,true,true)
def threeIn100Years = Seq
.fill(100)(coinFlip(1.0/500))
.containsSlice(targetEvent)
// Simulate this 100,000,000 times
val simulations = 100000000
val count = Stream
.fill(simulations)(threeIn100Years)
.foldLeft(0){case (acc, cur) => if(cur){acc + 1} else {acc}}
// Estimated probability for a single city.
val p = count/simulations.toDouble
// The probability it will happen *at least* once in a population of
// 100 cities.
// This calculates the probability if *won't* happen 100 times, and
// then takes the complement, giving us the final probability
1 - Math.pow(1 - p, 100)
The final answer I get is 9.5E-5, so still very unlikely, but 100x more likely than the binomial calculation.
I guess what I meant wasn't "non-independent events". However, there is still a fundamental difference. The probability of a 500-year flood happening next year is still 0.2%. With a birthday paradox, the filling of birthdays is what causes the explosion in probability. Assuming no collisions so far, the 20th person you add to the room has a 19/365 chance of colliding, while the 2nd person only has a 1/365 chance.
I'm not sure how to succinctly phrase this idea. Maybe "non-constant probabilities"?
If you put 400 people in a room, the odds that no two of them share a birthday have gone all the way down to zero percent.
No matter how many independent cities you measure flooding in, you can never achieve a perfect 0 or 1 probability of (whatever), because the cities are independent.
We don't define "interesting" flooding in one city by reference to what happened in another city, so this situation is not described by the birthday paradox.
> Yet, if you have 30 people in a room, there is a greater than 50 percent chance that at least one PAIR has the same birthday.
Not quite the same because Huston is not the only city that has experienced extreme climate these two years, many other cities also have (maybe not 500 years flood, but close).
So that would be like: one pair is born the same day, and 20 other people in the room are born a week around that day.
Hardly a coincidence.
It's more likely that there was a party about 9 months and X years before that day. (I'm not sure how far we can go with this silly birthday analogy:)
I think that this statistic only makes sense if it is specific to the location. Otherwise, it is useless for planning purposes. If it is specific to the location then the birthday paradox does not apply.
Interesting note, this is Houstons third 500-year flood in three years [1].
> it's actually quite likely that at least one of them could have a pretty high probability of experiencing one in 500 chance of something each year.
This is the key point here, and it's called the fallacy of multiple endpoints. If there are 500 observable cities, then somewhere among them should occur a "500-year event" every year. It's just that nobody ever notices the big bulk of the population that hasn't yet had any 500-year event in its recorded history.
How do you count "floods"? If there is one hurricane that causes floods in 3 cities, is that 1 flood, or 3?
If it's 1 - which is what I think is meant when saying "a 1 in XXX years flood" - then your comparison doesn't hold, because the number of floods isn't dependent on the number of cities, but on the distribution of cities on the land, and specifically in flood-prone areas.
IMO you need to look at something other than city limits. Cities like Houston and LA are huge cities geographically. IMO you need to look at regional and per capita impacts.
Ahem, no - your birthdays example can't apply here. When you increase the number of people in the room, each one of them benefits from every new person added, in terms of probability of having been born on the same day.
In the flood case, a flood in California and one in Miami don't really increase the chance of another flood simply because they have happened.
With water? Unlikely. Water doesn't just drop and disappear on the same spot in the middle of nowhere, it tends to move downstream, into places that are in the middle of somewhere - as we have seen with the Mississippi watershed: first Out In Nowhere, IN, then ORCS, then Lake Ponchartrain. A 500y water event in nowhere land would still propagate; the speed of water release from the drop point (sorry for not using the correct jargon) would mean the difference between a 10y event downstream or a 500y event on the go.
It's more analogous to getting black 17 six times in a row. It is possible, but it's weird enough to justify thinking that you've misunderstood the game, or the game has changed.
The "Birthday Problem" has a simple explanation. Something like nearly 45% of people are born in the Summer months. Ensure that you get people of different ages and your chances of finding someone with similar/same birthday goes up enormously.
No, the birthday problem still works if you consider that the probability of being born on a particular day is uniform.
It's related to the fact that the probability of two people in a room having a birthday is more closely related to the number of pairs of people in the room (which in turn grows as O(n^2)), than to the number of people in the room.
A detailed proof/explanation can be looked up easily with Google, if you have a bit of mathematical maturity.
For those who down voted me, you do not know your sample set or your test subjects. Go to the empirical data sources. Human behavior certainly affects the real probabilities. As of 2015 in the US [0], a person is more likely to be born in July, August or September. They're also more likely to be born during the middle of the work week rather than the weekends. Draw your own conclusions from the data.
Probability is just a model in lieu of the actual data with many built-in assumptions. Probability is a shortcut finding, collecting or generating data. My point is, if you have the data or a very large faction thereof, use it. Don't guess using probabilities.
The birthday problem still exists even assuming that everyone has an equal chance of being born on any day of the year. It's a phenomenon of the statistical likelihood that given a large enough group of people, the likelihood that any two will have the same birthday goes to almost certainty very rapidly.
Is this true? Which months are considered "summer"? Northern Hemisphere or Southern or both? This can easily turn into a "40% of sick days are on Monday or Friday" type fallacy.
But storms aren't people with randomly distributed cities to flood so you don't get the combinatorial effect. It's not like there are a fixed number of 500 year storms every 3 years that have to flood a uniformly distributed list of cities. The floods are supposedly independent (even if our estimates of likelihood are bad) so there's no binomial coeeficient.
The simple calculation is P=1-(1-1/500^3)^N and the probability is less than 1:10000 assuming independence and even given 6000 cities of a million people (the entire world population is in a similar city).
If it was birthday like (every possible 500 year storm for 3 years is in a room choosing out of 100 random million person cities), the probablility would be reasonable. You can use a Poisson distribution to estimate the 3 person birthday problem. P=1-exp(-3choose100/500^2)=48%
Actually, in the US it is approximately 1 in 250,000 /assuming independence which is a bit of an assumption/. Obviously, nature doesn't know what a year is so it is totally possible that big storms come in batches.
Assuming 1 in 500 year floods are independent, note the ~300 cities in the US [0]. The fact that Huston is a big city doesn't seem relevant to me, because the storm doesn't care. Much like picking a specific number plate, it had to land somewhere.
The chance of there being a city in the US which has a 1 in 500 year flood is therefore 60% (300/500) each year.
The chance of that city having another flood of similar magnitude in Year 2 is 1/500, because the odds are independent and we started monitoring the city in Year 1. The chance of it flooding again the third year brings the odds down to 1/(500 * 500) = 1 in 250,000.
> Climatologists say the mechanism by which this is happening is fairly straightforward. “Warmer air can contain more water vapor than cooler air,” according to the 2014 Climate Assessment produced by the U.S. government. “Global analyses show that the amount of water vapor in the atmosphere has in fact increased due to human-caused warming. This extra moisture is available to storm systems, resulting in heavier rainfalls.”
This is just so totally obvious. But I've rarely seen anything in news coverage about increased mean atmospheric water concentration. Also, increased water content is (of course) the expected multiplier for increasing CO2.
A similar thing has happened on the Mississippi river, which has had '500 year floods' in at least 2011, 2009, 2008, 1993 and 1927. I may be missing a few.
Assigning probabilities to rare events is meaningless unless you a) have data over a much longer time period than the recurrence time and b) the mechanism causing flooding is not changing.
Neither of these holds in the case of Mississippi flooding, for which we only have about 150 years of observations.
Unfortunately, multi-billion-dollar decisions about flood risk are made using these manufactured numbers. For example, in Missouri, land behind a 100-year levee is not required to have flood insurance.
"Assigning probabilities to rare events is meaningless unless you a) have data over a much longer time period than the recurrence time"
This is probably the most important factor nationwide in the US. There simply aren't written records of the weather in the vast majority of land in North America which cover the past 500 years. I wouldn't even be surprised if more than half the land in NA didn't have meaningful data over 200 years old.
"b) the mechanism causing flooding is not changing."
This is probably the biggest factor in Houston's extraordinary reoccurence of "500 year" floods. While Harvey's level of rainfall is truly unprecedented in all of American history, the other floods can easily be blamed on the fact that the rapid growth of Houston's sprawl has turned a gargantuan amount of land area into a concrete swimming pool. Like other posters have mentions, undeveloped land is very effective at slowing and absorbing water but a gutter is nothing but an aqueous superhighway. Flooding isn't just caused by the total volume of rainfall, it's caused by the speed with which that volume reaches natural outflows (rivers, creeks, estuaries, etc).
Should any land be required to have flood insurance?
The NFIP supposedly reduces taxpayer cost (with the shaky risk management you mention), but it currently has huge potential liabilities compared to the premiums it collects (and owes the government a bunch of money from past payouts).
Hey we've got a legal mandate to fund insurers now with our 'public private partnership' healthcare system.. pick this up as your issue and you might have a lucrative career in lobbying lined up..
The idea was and is twofold: first, it was seen as the only possibility to actually provide such insurance because insurance companies (even together) lack the financial strength to be able to insure risks that are statistically extremely dependent. Something like this flood costs dozens of billions, and it would bankrupt the whole industry.
Secondly, it's in the national interest to promote such schemes that allow for more areas to be used productively.
Maybe we should change the nomenclature, 500-year flood denotes once every 500 years. As opposed to a 0.2% chance. This, in turn, causes people that don't pay attention to floodplain maps to say, "we had a 500 year storm last year, we are good for another 500 years".
That, my friends, is a prime example of the gambler's fallacy.
The nomenclature is not the problem, the problem is making numerical estimates of low probability events based on just 150 years of data and a raft of modeling assumptions.
The problem is not making numerical estimates of low probability events or how such events are modeled: it's completely ignoring the statistical probability distributions of the model. All of the models are extremely "long tail" distributions and just about entirely ignoring the long tail.
We shouldn't be referring to this as a low-probability flood but as a high sigma flood.
ETA: Disclaimer: Day job includes rainfall statistics analysis.
I don't know much about the field, but I'm curious. How much are estimates based on normal distributions, and how much does the model consider other distributions?
As you say, I don't think we know much about the tail. More than ten inches of rain may be a once a year phenomenon, but that once a year event might be twenty or fifty inches?
Disclosure: I am not a subject matter expert, I just do the programming they tell me to do. Mistakes/oversights here are likely my own, not my employer's or the software I work on.
The rain gage analysis the software I work on does is largely based on USGS data. [1] Almost all of that data is publicly accessible and you can explore the data down to individual monitoring stations if you wander through the site far enough.
The application I work with is primarily concerned with two bits of analysis from the data in a given station: average peak annual flow and mean daily flow. The distribution used for analysis of both (beyond linear interpolation and best fit line options) is fitting to a Gamma distribution [2], and plotted on a logarithmic scale. (Rainfall is specifically mentioned under applications of that distribution on Wikipedia, so it seems to be the industry standard even outside of the specific application(s) I work on.)
I am not a math, but I remember that if you can't assume a normal distribution and want to know the shape of the tails, you need a pretty ridiculous number of observations.
I don't think it's accurate to say we only have 150 years of data. We only have about 150 years of data collected in real time by humans, but we have plenty of other data. (Soil/rock strata,core samples, tree rings, etc.)
In some ways the geological and archeological evidence is more trustworthy than the evidence we've written down in the past century or two.
Do floods show up on the "geological and archaeological" records? Sure we can say that 50M yrs ago there was an ocean here, but that's a different statement than "in the last 1000 yrs there have been 14 floods that reached this height above the river".
No, it's not so simple either. If 200 years ago there wasn't a city there, then any kind of soil/rock strata, core samples, tree rings etc. likely won't show flooding, and therefore won't be representative of flood risk today.
Houston is a concrete jungle, which significantly increases the risk of flooding compared to even a decade ago.
Honestly I think it's easier to feel irrationally safe about a 0.2% chance, especially if you leave out the "per year". "500-year flood" at least emphasizes that it's not a question of "if", but "when".
There is also a feeling that a 100-year event is too rare to be concerned about, though I don't know that changing it to a 1% per annum would change many perceptions.
The elephant in the room has been mentioned somewhere else these days: severe lack of planning and crumbling infrastructures.
Remarkably how ne of the few non-partisan issues on the table goes under the radar time and time again. Tongue in cheek, we might think only polemic issue get the attention of the public.
There is also the notion of national pride. We built our infrastructural base earlier than pretty much anyone else in the world, but we don't invest consistently on them anymore. Today our roads, dams etc. pale in comparison with any western country's but it seems like it's an affront to our national pride to acknowledge that.
Houston is ill prepared to face any kind of emergency like this one, and only the bravery and tenacity of the people is preventing a humanitarian disaster in the scale that would keep us awake at night. That's not the American way: planning, preparing and executing is. The sooner we start taking this leap the better.
Which other city in the US can handle 40 or 50 inches of rain, and 15 trillion gallons of water, within a one week timespan? That's as much rain as most US cities get in a year.
This is not the fault of poor planning or poor infrastructure in Houston. It's freak weather event that shouldn't just be given partisan rhetoric.
Probably none, but it's not a matter of what city could survive unscathed, it's a matter of what city has minimized the consequences. Houston has allowed for basically unfettered growth constraining their bayou's and preventing flooding infrastructure to be built. Hopefully Houston uses this disaster to rezone and allow for appropriate land management.
Similar freak weather happened 16 years ago in Houston (Allison, $5 billion in damage). It took Houston at least 5 years to recover then. It's pretty clear now that between the upgrades in water management after Allison and the addition of another 2 million residents since then, Houston wasn't as well prepared as it should be for another Allison, much less a Harvey.
In the past two consecutive years, Houston has experienced significant urban flooding due to what are now "typical" big rains. But even 1-year storm events like those were too much for the city's drainage system to handle. The city fathers must be aware that flooding has become a chronic problem.
The lesson from Katrina was that New Orleans' levees were undersized and unable to withstand overflow, leading to catastrophic flooding, and the destruction of homes not built to a modern flood-resistant design standard.
I think the lesson from Harvey was not that Houston has a severe drainage problem, but without significant upgrades, the problem is going to guarantee more ongoing and occasionally catastrophic floods in the very near future.
> The big news in the Galloway report, based on a global review of scientific and engineering data, was this: All of the money the United States had paid for massive public works to control flooding over the previous half-century not only had failed to improve flood safety at all, but the spending and the big public works projects actually were making flooding much worse.
> The Galloway report was no global warming screed. Like the Dutch national policy based on many of the same findings, it dealt almost entirely with land use and population growth. Or to put it in more accessible terms, the 'burbs.
Anyone notice the comments for this article on WP? There appears to be a poster called "The burning bush" who is commenting with lengthy comments about every 6 minutes. All of the the comments from that user are sceptical of man made climate change. Washington post does not appear to let you sort by anything other than "newest"... which means that this commenter's posts are always the top comment. That's concerning!
That's normal. You'll find the same thing going on in the HN comments for '24/192K audio is silly': there's a class of people who fight for their arguments not with science or discussion, but through brigading and suppressing other people's arguments to get 'em out of sight. It's very much the case with climate change denialists, and it's also in play with '16 bit audio is enough for every possible person!' audio pundits: they will brigade to suppress opposing arguments to make it seem like their views are unopposed.
Pretty sure the WP comments from the climate skeptics are more organized. There's a lot more money in climate denialism, from traditional energy companies with enormous resources and a clear agenda to continue their present mode of operations. I've never understood why the 16-bit audio guys are so similarly quixotic.
Every comments section I've seen for newspaper articles even in supposedly blue urban areas contain an outsize proportion of conservative trolls. It's possibly worse than youtube comments.
WaPo is experimenting with a new comment system, which is why it currently only lets you sort by newest. The old system had other sorting types, and the new one will too soon. Don't worry about it too much. :)
The problem is that the person who judges the probability at 1-in-500 doesn't have any skin in the game.
The judgement of the US meteorological office affects where people can build houses and how much they pay for government insurance. Homeowners have a strong financial interest in underestimating the probability of a severe flood, so there is likely subtle political pressure applied to the met office assessments.
For a more accurate assessment, the government should buy a sizeable insurance policy from several reinsurance companies and use it to calculate the implied probability.
Homeowners have a strong financial interest in underestimating the probability of a severe flood
I don't know about that. Even if I have flood insurance, it's not clear I'm financially better off by having my house flood. It may be a confluence of "financially not fully liable" and wishful thinking, though.
And the reinsurance folks seem to say that there is absolutely no evidence that there has been an uptick in these big disastrous storms. People are constantly thinking of global warming, so when they see a storm, they say "Aha! Global Warming!". Of course there have always been hurricanes, and human perception/remembering is not a replacement for statistics and math.
From Warren Buffet (One of the largest insurers of this type in the world) "...climate change had not up until then [2016]
produced more frequent nor more costly hurricanes nor other weather-related events covered by insurance.""
Bad shit happens when you fuck up the earths temperature. Why is anyone surprised? This has been said by climate scientists all over the world many times.
Warmer climate will result in more drought on some places and other stronger storms which result in heavy rainfall and floods.
Fort Bend County Judge Robert Hebert stated that Hurricane Harvey is more like a 800-year event, as quoted from the statement from Monday, reported by the same newspaper WaPo[1]
"Fort Bend County Judge Robert Hebert stated that Hurricane Harvey is more like a 800-year event"
500 years, 800 years ... even 1000 years - none of those are timetables that planning for city ^H^H^H^H region the size of Houston should be operating on.
A city that large, that populous and with that much economic activity that is crucial to the world should be using risk models to 10k year events.
This is not unprecedented - Holland's dykes are designed and built to withstand 10k year events:
There was an article from 2016 also on HN today about the risk. Three different experts were quoted saying something along the lines "nothing will be done now, we need to wait for the big flood and 2-4 years after we will build a wall".
The "wall" (actually a more complex thing) was projected to cost 8 bil. A bad flood like the one from today was projected to cost 70+, but people were hoping that we have around 10-20 years to prepare
The City of Houston, which votes Democrat, has actually done a reasonable job of upgrading its infrastructure over the years. For example all roads built/upgraded since the 90s are designed to act as drainage systems.
The problem is that Houston is a big city and its population has been rising very quickly, and the city government's efforts have been insufficient. And since the Texas state government is dominated by gotta-keep-government-small-and-taxes-low Republicans, Houston has received almost no assistance from the state in this regard. We are seeing the results of that today.
Coastal protections are under the purview of the Army Corps of Engineers, a federal institution, one of the most sclerotic of the government institutions, and one of the rare beasts that actually deserve the common cynicism with government action.
There is no way to credibly make this kind of claim. We don't have the data to quantify the recurrence rate of low-probablity events like this, nor can we be sure the the underpinning mechanism isn't changing with time.
I don't think I understand what you're saying. To be able to say things about 0.2%/year events, we would need many thousands of years of data on an unchanging system.
And yes, basically we don't understand the actual probability of a "500-year storm", so our 0.2% chance per year is unreliable at best, more likely misleading.
Just like Europeans didn't know the actual probability of a black swan until they found them in Australia.
I think the idea of a black swan is something which you don't even imagine before you encounter it, ie: nobody would even think about computing the probability of a black swan (bird) existing.
While unimaginability may fit the original conception of a black swan, Taleb's book is "The Black Swan: The Impact of the Highly Improbable" (not unimaginable or inconceivable) and defines a black swan as "an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility." http://www.nytimes.com/2007/04/22/books/chapters/0422-1st-ta...
This article reminds me of Nate Silver's book [1] which has a far more scientific and in depth look on the failure/success of these types of predictive statistics.
The chapters on the many attempts (or many failed hopes) in predicting earthquakes was particularly interesting, including many times the media has bought into hyped up new charlatans who say they finally figured it out but which ultimately failed to survive under basic statistical scrutiny.
It also has a useful soft introduction to Bayesian statistics and other useful concepts from the field of prediction that I hope more journalists read about. As this seems to be a very common theme in reporting.
Even this journalist couldn't help themselves with this line (combined with some scary looking charts described with an alarmist tone farther down):
> Was there some miscalculation of how frequently these massive flooding events occur? Or, most alarmingly, is something else happening that suggests these catastrophic weather events are becoming much more common?
The failure to mention the effects of El Nino/El Nina seems like a big oversight in this article, especially when we're just coming out of a particularly strong one. Climate stats are an easy one to get wrong - or to shape into any narrative - especially when timeframes and location are easy things to be viewed too narrowly.
Howdy! I work in cleantech, and I guess it's that time again for a what-can-you-do-about-it post :)
To start, here's my favorite climate change joke: "They say we won't act until it's too late... Luckily, it's too late!"
==So what can you do about it?==
The biggest thing that is most relevant to the HN audience is that you can work at a new energy technology company! Our industries are out of the R&D stage and are currently focused on scale and growth[1], and we need as many smart people as we can get. There are lots of companies hiring software engineers.
==How do I find a job fighting climate change?==
I'd recommend browsing the exhibitor and speaker lists from the most recent conference in each sector (linked below). Check out the companies that interest you and see if they are hiring.
* Energy Storage[2][3]
* Solar[4][5]
* Wind[6]
* Nuclear[7]
* Electric Utilities[8][9]
* Electric vehicles[10]
Also, if you're in the SF bay area, I'd recommend subscribing to my Bay Area Energy Events Calendar[11]. Just start showing up to events and you'll probably find a job really quickly.
The alternative approach to working directly towards solving the problem is to work at a mature tech company with higher pay, and donate 10% or more of your income every year to a non-profit like the Natural Resources Defense Council.
Unless you're a lobbyist I feel like donations to lobbying organization would have the greatest impact. Many politicians are bought for what is comparetively a small amount of money compared to the impact they have.
Talking about Cleantech, if you live in California you can signup for Solar Choice from PG&E. It charges you extra for electricity, with the proceeds going to a program that offsets your electrical consumption by building new renewable energy sources. Here's a link to get started: https://www.pge.com/en_US/residential/solar-and-vehicles/opt....
While energy is definitely a huge part of the problem, preventing things specifically like Houston's flooding seems like more an issue of infrastructure and emergency readiness.
And dealing with the aftermath seems like it's a space for organizations dealing logistics, drone fleets (e.g., for mapping, finding survivors and communicating with them, delivery to inaccessible areas), etc..
I saw a TV programme a year or two ago which was discussing the increase in hurricanes etc in the Gulf of Mexico area.
They were saying that the storms aren't increasing due to global warming but are actually regressing back to their natural levels. The previous decades of pollution had been suppressing the storms and by cleaning the air the storm ferocity is heading back to normal.
I can't see the article at the moment but I think it was Cloud Lab on the BBC, they floated around on an airship.
The fundamental takeaway from this article is about half way down. Major flood events are occurint more frequently. What was once likely to hit a city every 500 years is happening to more cities on a planetary scale more frequently.
I know it's easy to blame climate change for anything and everything, but that's simply incorrect.
The actual takeaway is that our data is wrong, these floods occur more often than we thought. With only 150 years of good data, it's impossible to calculate "500 year" events accurately. You would need around 5,000 years of data to do that, and we don't have it.
So we ended up significantly underestimating how often these events occur.
You can see in graph that floods are occurring more often and that clouds are able to absorb more water as temperatures rise. This hurricane would have happened. But this flooding is worsening because of climate change that is irrefutable.
I feel sad when people use statistics like this. It makes no sense to call this 500 year flood because floods do not follow any kind of distribution. Not to mention the human beings have been changing their environments drastically in last 200 years.
NN Taleb would say this is fat tail event. It does not follow empirical distribution.
Of course, the "once in 500 years flood" may also be an inaccurate probability as well, but if its not, then I'm not surprised this is happening SOMEWHERE.