While it can't be done server-side, this can be done straightforwardly in a signer service, and the signer doesn't need to interact with the payloads being uploaded. In other words, a tiny signer can act as a control plane for massive quantities of uploaded data.
The client sends the request headers (including the x-amz-content-sha256 header) to the signer, and the signer responds with a valid S3 PUT request (minus body). The client takes the signer's response, appends its chosen request payload, and uploads it to S3. With such a system, you can implement a signer in a lambda function, and the lambda function enforces the content-addressed invariant.
Unfortunately it doesn't work natively with multipart: while SigV4+S3 enables you to enforce the SHA256 of each individual part, you can't enforce the SHA256 of the entire object. If you really want, you can invent your own tree hashing format atop SHA256, and enforce content-addressability on that.
I have a blog post [1] that goes into more depth on signers in general.
Most jurisdictions make a distinction between moving violations, which are issued to the driver, and parking violations, which are issued to the car (and its owner, by extension). This is why, in most places, you cannot get points on your license from parking illegally.
This situation would have been a moving violation. It sounds like the law has not caught up with the concept that a company might hold a driver's license and be issued moving violations.
Yes, by design. One major goal is to prevent a landowner from squatting on an empty lot while their neighbors build prosperity around it. The "squatter" then cashes in, having done nothing themselves. "Everyone works but the empty lot" is the commonly used phrase.
The goal of an LVT is to insulate a landowner's tax bill from being affected by their own improvements. Its anti-goal is to insulate a landowner from changes in land use around them.
Another benefit is that it encourages more efficient use of land. Existing taxes mean that if you demolish a high density retail strip and replace it with a McDonald’s or Walmart with large parking lot/drive thru, the tax income plummets while the cost to maintain the roads and plumbing remains the same.
Just to be clear: land value taxes by themselves don't encourage more efficient use of land. Removing taxes on improvements (and on capital and labour) encourages more efficient use of land, and a land value tax can help finance those other tax reductions.
The two downsides I'm aware of are difficulty in transitioning to an LVT and difficulty in valuing the land.
Transitioning to an LVT means that landowners no longer capture land rents for themselves, which is a massive overnight loss in the value they hold. The solutions there typically tax only the difference in land value versus a baseline assessment. So if a lot is worth $100 before LVT and $105 after, the tax is calculated only on the $5 difference.
Valuing the land is tricky because the whole point of LVT is to tax only the location itself. So the value of any structures should be excluded from taxation, and even improvements in soil quality (e.g. on a farm) should be excluded. This is problematic because the market for bare land is significant less liquid than improved land, especially in suburbs and cities. So there isn't always good data on comparable land, and there isn't a way to hold a straightforward auction to value a given lot. Of course, most present systems of property taxation are subject to the exact same issue.
- There are deeper problems with valuing land. Land values in cities are directly related to approved zoning (i.e. what you are allowed to build), so the city government can rezone neighborhoods and unilaterally alter the land values the residents pay tax on. This may not agree with everyone's view of fairness.
- LVT encourages building tall and is hostile to lowrise development and unbuilt/green spaces. Those policy preferences may not be shared by everyone.
>> Of course, most present systems of property taxation are subject to the exact same issue.
This is not really true. There are constant sales of building+land in cities and estimating building+land values can reasonably be done.
In a city bare land almost never trades.So you have to extract land values from building+land sales, which is much much harder and possibly impossible to do fairly.
> - There are deeper problems with valuing land. Land values in cities are directly related to approved zoning (i.e. what you are allowed to build), so the city government can rezone neighborhoods and unilaterally alter the land values the residents pay tax on. This may not agree with everyone's view of fairness.
That's actually a feature, especially if you make sure that the authority who can do the zoning also gets the revenue (or at least shares in it). That way aligns incentives.
> - LVT encourages building tall and is hostile to lowrise development and unbuilt/green spaces. Those policy preferences may not be shared by everyone.
LVT doesn't do anything like that. The whole point of LVT is that it has no influence on land use choices: you literally pay the same LVT no matter how you use the land. It doesn't encourage or discourage anything. That's why it is economically efficient.
(However, alternative taxation schemes like income tax or capital gains tax or taxes on improvements do discourage building tall. And if you lower those taxes, people will build taller.
Btw, I think that for all its faults a conventional property tax that doesn't distinguish between land and improvements is still miles better than income tax or capital gains tax or sales tax etc.)
> In a city bare land almost never trades.So you have to extract land values from building+land sales, which is much much harder and possibly impossible to do fairly.
Often land changes hands and the new owner tears down the structure and build a new one. You can reasonably assume that the old building was valued at zero, or even negative because tearing down costs money and time. So that gives a lower limit on the price of the bare land.
>> That's actually a feature, especially if you make sure that the authority who can do the zoning also gets the revenue (or at least shares in it). That way aligns incentives.
This is only a positive if your goal is to upzone everything. If you think cities should be a mix of zoning and zoning shouldn't be driven by tax considerations, then this is very negative, since the land management department has an incentive to increase zoning and taxes.
>> LVT doesn't do anything like that. The whole point of LVT is that it has no influence on land use choices: you literally pay the same LVT no matter how you use the land.
I don't agree that's how the incentives work.
If you don't tax structures you absolutely incentivize building structures, because they earn money but pay no tax.
In a land+building tax structure, there is less incentive to build a structure because they pay tax.
If I have a lot of green space and few structures, and we convert to LVT, I will be taxed proportionally higher than before, or than my neighbor with less land and more structures. By taxing me more you are dis-incentivising my approach.
Well, I think zoning is mostly silly, and people should mostly be able to decide what they want to do with their property.
(Before zoning was a thing there were already nuisance laws that forbade opening heavy industry next to a Kindergarten. No zoning required.)
In any case, people don't build high rises in the middle of nowhere right now. They won't start (or at least not much more than under the status quo) if someone drops taxes on structures a bit.
Also keep in mind that people don't get spontaneously generated. If people cluster together to form a high density area, some other parts of the country will see lower density. Ie if you let all the people who bunch up together, bunch up together, there's more space left over for the people who prefer lower density.
> If I have a lot of green space and few structures, and we convert to LVT, I will be taxed proportionally higher than before, or than my neighbor with less land and more structures. By taxing me more you are dis-incentivising my approach.
What you are describing is purely an effect of whether you tax structures or not. It's completely independent of whether you tax the land value.
> Land values in cities are directly related to approved zoning (i.e. what you are allowed to build), so the city government can rezone neighborhoods and unilaterally alter the land values the residents pay tax on.
Cities were already able to rezone neighbourhoods and unilaterally alter the values of residents' land (also just through everyday building - if they build a transit station in one neighbourhood and a sewage treatment plant in another, that alters everyone's property values), and this was already a very corruptible process. In theory LVT should improve it a little since now the city has an incentive to increase everyone's land value as much as possible.
> LVT encourages building tall and is hostile to lowrise development and unbuilt/green spaces.
Yes and no - it encourages making valuable use of expensive land, and moving less valuable uses onto cheap land, but it's agnostic about what that "valuable" is. If people prefer - that is, will pay more to use - lowrise buildings or green spaces, then that's what LVT will deliver.
It's rather easy to value the land: Have the owner decide what it's worth, then they pay a tax as a percentage of that valuation.
Now, obviously given that system everyone's going to value their land at $0.
To adjust for that land owners must be obligated to sell their land to anyone willing to buy it at the declared valuation.
Such a mechanism doesn't only keep the current owners honest, but leads to more accurate price discovery, as the land might have a higher "real" valuation than the current owner is aware of.
I don't love this idea, and I suspect many others wouldn't either. If you raised your family in a house and have lived there for decades, it has intangible value to you, but not others. Yet because of this value, you must pay potentially much much more than your neighbor, who objectively speaking may have a lot of equal value.
What's more, even if you pay more than you ought to, you'll never feel secure in your home, knowing that at any time you may be forced to sell.
You can pull on those heartstrings in either direction, as it were.
In economic terms what you're arguing is that investment efficiency should always outweigh allocative efficiency.
> pay [...] much more than your neighbor, who objectively speaking may have a lot of equal value.
All land is unique, so I don't think adjacent land of equal value exists. The difference may be trivial, or it may be substantial.
But yes, it's all a tradeoff. Some might prefer a centralized government authority decreeing a given value, others might prefer market-based price discovery.
I'm not trying to convince you or anyone else either way, just pointing out that fair price discovery for a self-assessment LVT isn't an unsolved problem.
> All land is unique, so I don't think adjacent land of equal value exists. The difference may be trivial, or it may be substantial.
I don't understand how this relates. My point would stand even if the neighboring lots were slightly different in value.
> just pointing out that fair price discovery for a self-assessment LVT isn't an unsolved problem.
Yeah okay I'll give you that. It's just that we can't ignore how tax policy must match a society's values in a democratic society, else it'll be voted out. I'm saying this probably wouldn't work out since voters put value on the idea that at least some people will be able to get a good enough job to afford to bring up their kids in a stable home.
I'm agreeing with you (along with the "heartstrings" comment) that all land is going to have both objective and intangible value, e.g. the view, and that someone grew up in that house.
But I think you're imagining that any intangible interests in the land are going to favor the incumbent.
I think for residential lots that's probably more true than not on average.
But we can easily come up with examples where a prospective buyer has a stronger intangible interest.
E.g. maybe you own it, and don't really care about the land or house per-se, but it saves you 1 minute on your commute v.s. the next lot.
Whereas I used to live there, and was forced to sell the house during the last recession. I've got a deep emotional connection to the lot and house, and my dog's buried in the backyard.
I'd like to buy the house back. You don't want to sell.
Does my interest outweigh yours? Maybe, maybe not.
All I'm saying is that a self-assessed LVT with an auction mechanism (see https://news.ycombinator.com/item?id=37909570) will enable both of us to set a price on those intangibles.
I agree that probably nobody's willing to try this out any time soon, for what it's worth the authors of "Radical Markets" suggest phasing in such a system by starting with commercial lots (and perhaps it would never go beyond that).
Indeed, and the mechanism for adjusting any unfairness in assessment you encounter there is always going to be more byzantine than a fair self-assessment, and probably impossible in practice.
E.g. let's say you live in a neighborhood where everyone's paying a premium for fanatic views. Except your house is the only one that doesn't have that view.
Even in such an obviously unfair scenario the government is likely to stick to some assessment that's going to be unfair, e.g. some mean sale value of the N lots adjacent to yours.
Also with land value tax without the proposed land valuation method. The difference is how in one case you're never secure in your home even if you can afford your theoretically fairly assigned tax bill. There is always someone out there rich enough to uproot a family from their home, even if you're otherwise financially secure.
Your weird scenario can happen with most other assessment scenarios either.
If Warren Buffett wants to increase the value of my land, he can bid up all the surrounding plots, and make bids for my land. Any sane assessment method will see that the value of my land has increased, and will increase my property tax or LVT, and I'll have to pay or face the consequences.
(In the self-assessment case, you can give people the right to refuse to sell, if they are willing to eg back-pay the difference of LVT to the higher price for the last year or so. So people can opt to pay the tax instead of moving out.
To be extra fancy, give the would-be-buyer 1% of the extra tax take to incentivise people hunting for undervalued homes and to compensate for the buyer having had to secure funding.)
I don't see that as a fair comparison. The number of people who are rich enough to buy a particular home for more than the hypothetical actual value is orders of magnitude greater than the number of people who can afford to buy up all the lots around a home at a premium and affect its property value. As a result, the perceived threat to the homeowner is substantially different. I also think you're ignoring the psychological costs of this. It doesn't actually have to happen, just the potential that it could happen would be a real fear for many people. Moving is one of the most stressful events in people's lives, much less unplanned moving. There's also empirical data showing negative outcomes for children.
If that kind of fear is a problem for people, it will just be reflected in lower market prices for land.
Someone who has more of a fear for can put up her self-declared land value. They'll pay a bit more in recurring tax, but would get a significant windfall, if their fear were to come to pass: Yes, there might be some psychological downsides to moving, but getting a extra few million dollars (or whatever) has psychological upsides, too.
I'm from the younger generation who's been economically forced to move every couple of years because the older generation made it too hard to build houses. So I'll shed few tears for boomers being forced to move once after 20 or 30 years.
House prices don't change at the drop of a hat. If gentle changes in taxation are too much for a person, they might just have purchased a house they couldn't afford in the first place.
Given the friction (inconvenience) in having to move house/business etc, that hardly seems fair. It also defeats the object of land ownership if you can be kicked off it at any time.
You can trivially structure such a tax in such a way that going through a "forced sale" is going to be very lucrative (edit: elaborated in a sibling comment: https://news.ycombinator.com/item?id=37909570).
> It also defeats the object of land ownership[...]
So no, most people could keep land they'd like to keep in practice.
The entire notion of any sort of property tax is also predicated on the idea that individual land ownership is a tradeoff between the interest of the individual and society at large.
> The entire notion of any sort of property tax is also predicated on the idea that individual land ownership is a tradeoff between the interest of the individual and society at large.
Indeed, and I like this concept, I don't think we should ever "own" land in the same way as I own, say, my phone. All we ever do is borrow it from society (or even nature).
However, some aspects of land ownership are a net positive for society at large. In particular, the incentive to look after it better if it's really yours until you sell it or you die. With this in mind I like some proposals I've heard whereby unpaid land value tax can be accrued to be paid at death or on sale. That way the stewardship aspect of ownership is reinforced without the freeloading on land value increases.
Returning to the valuation question. I think you're assuming an efficient market when it clearly isn't one. It would be like having to reapply for your own job, except it's rebidding for your own house. Not a kind thing to do to anyone.
This does not pass basic scrutiny unless the plan is to ruin average homeowners that valued their assets at fair market value.
Essentially, you want to force asset owners to write an at-the-money call option against their assets, and then adding insult to injury by not paying them an offsetting risk premium. I don't know how any moral person could be a proponent of the kinds of abuse and profitable exploitation of average people this proposal would trivially enable.
Yes, you would be forcing people to write a call option. It doesn't have to be at-the-money. Owners just pick a price that they'd be happy to sell at. Not some mystical 'fair market value' that would ruin them.
Of course, land owners would want to keep their tax bill low, so picking the right price to declare is a trade-off.
> [...] and then adding insult to injury by not paying them an offsetting risk premium.
Please be more careful in your reasoning! You are right that the call option is worth a premium. But that obligation to write the call option comes with ownership of the land, so we can just treat it as another (small) tax on the land. The market price of the land adjusts so that the yearly benefit from owning the land is pretty close to the yearly cost of capital plus sum of all taxes.
To simplify: the option premium is automatically offset by lower LVT payments.
> I don't know how any moral person could be a proponent of the kinds of abuse and profitable exploitation of average people this proposal would trivially enable.
Please elaborate. But please refrain from assuming that landowners are morons.
In particular the book by Glen Weyl mentioned in that article describes how it could work in more detail, and in a way that address the concerns you have.
A relevant except from that book (which I've got a Kindle copy of):
> For any tax rate below the turnover rate, the possessor will always set a price above the amount she is willing to accept[43]. When the tax rate is zero, the possessor is free to set any price she wishes at no cost and thus would set the monopoly price. When the tax rate equals the turnover rate, she has to reveal her true value. For intermediate tax rates, she will still be discouraged by the tax from setting a very high price, but she will not have a full incentive to report her exact value. Instead, she will set a price intermediate between her true value and the monopoly price that she expects a buyer to be willing to pay. As the tax rises from zero to the turnover rate, the price she quotes will gradually fall from the monopoly price to her true value.
That 43rd footnote in particular further addresses your exact concern (the mentioned "COST" stands for "common ownership self-assessed tax"):
> 43.: This fact helps allay two potential objections to a COST: that possessors may wish to “sabotage” the appeal of their goods to others to avoid their interest in taking the good, and that predatory outsiders may maliciously take goods just to harm a possessor. Notice that neither of these are possible if possessors always set prices above the minimum they would be willing to accept, because in this case the possessor is happy when her possessions are taken: she still profits, just not as much as if she set a monopoly price. Thus “predation” will be nearly as welcome as would be the “predation” of someone offering you out of the blue an extravagant sum for your home and you would never wish to sabotage your possessions as this would reduce the chance of such an exceptional opportunity. Only individuals who fraudulently report extremely low values and try to dramatically sabotage their goods would be open to predation, but so they should, and such individuals are likely to be caught by others before too much sabotage is possible.
That’s a horrible idea - that means you value is only what you can afford to pay in taxes - effectively meaning the poor would be required to sell to the rich at below market rates if they can’t afford the taxes
Are you suggesting the rich people form a cartel? Otherwise, if there's more than one rich person they would outbid each other until market rates are reached. That's basically how market rates are defined.
Btw, none of the problems here are specific to LVT. You see exactly the same problems in conventional property taxes: if your land is suddenly worth a lot more, your tax bill goes up.
To me, the biggest downside is that the government is running this.
If the government turns on an LVT, do you trust them to turn off other forms of funding? Or do you think they're just going to decide that their income went up?
I kind of trust them, but I don't have very high confidence that they won't decide that they "need" the extra money, maybe just for some "emergency" situation...
The nice thing is that lowering other taxes automatically increases the LVT tax take via higher property prices.
You can see that dynamic on the border between Switzerland and Germany: Swiss income taxes are a lot lower, so their property prices are higher. (You can probably see similar things happening on some borders between American states?)
Without an LVT, those higher property prices only benefit the land owner. With an LVT, the government gets some incentive to lower those taxes.
> If the government turns on an LVT, do you trust them to turn off other forms of funding? Or do you think they're just going to decide that their income went up?
I guess it depends. Governments have an incentive to get themselves more budget, but taxes also aren't just ratcheting up all the time. Eg the US still has lower income taxes than most of Europe. And tax rates also change over time.
This seems by _far_ the biggest difficulty, and I find it strange that this rarely comes up in LVT discussions. Even for nominally 'liquid' land it's not clear who values it or how.
Lots of countries value land and improvements separately (e.g. New Zealand, Australia, Denmark) for council rates (property tax essentially). Here in New Zealand, there are several companies that provide valuation services to local councils (and private individuals if you want to value your property). You can dispute the valuation if you think you're being unfairly valued (usually because you want to pay less property tax).
It doesn't seem that different from what normally is done for property tax. Property tax is already generally based on land and structure (Just look at property taxes on the million dollar tiny houses on prime real estate). LVT is just removing the structure portion.
Is it? I can tell you now, I have no difficulty asserting that the value is near zero for the depreciating, crumbling houses on top of most of the land in the Mission in San Francisco. All the appreciation, which means nearly all of the sales price, of a typical home here, is the land.
BuT rEnNoVaTiOnS. Listen, I’m not trying to give you a comprehensive answer. I’m just trying to show that it’s not by far the biggest difficulty, not in the places LVT is most impactful, such as cities with extremely high vacancies like San Francisco.
The problem you're glossing over is what value should be assigned to the land.
If some land developer wants to build a new arena next to your plot of land - boom you're value just skyrocketed.
If the same land developer backs out of the deal - boom your land is worth less (or is actually worthless).
Your taxes depend on exactly when the assessment was made... and even professionals cannot agree on valuation (as we're seeing in some high profile cases right now).
Even for the same plot of land two people can value it radically differently.
Yes, and it's been long bemoaned about how people game that system and how unfair it has a tendency of being. Which was one of the points I was raising.
There is no objective valuation for anything really... particularly when it comes to more-or-less unique, speculative properties such as land and/or improvements.
The only reason everyone mostly agrees on, say a car's value is there's a lot of cars exactly like it that have been sold recently in whatever area you are in. Yet, every plot of land is mostly unique and has a tremendous amount of potential, debatable factors when it comes to value.
I'm a big fan of privacy.com: they provide me with a spend-limited debit card number that varies by vendor. I use them especially for newspaper subscriptions that make it difficult to cancel and have balloon renewal payments.
OP appears to be Jack O'Connor, one of the designers of BLAKE3, which is the fastest full-strength cryptographic hash function currently available. It's always nice to see practicing cryptographers also producing digestible cryptography content.
"fastest hash function" - I do not know much about cryptography, but isn't a hash function slow by design, or at least slow enough to prevent brute force ?
No but this is a common misconception. The key thing to remember is that a regular hash function and a "password hash" are very different. A password hash does need to be slow, because we want to make it expensive for an attacker who's stolen the hash to try to guess the password. But in most other use cases, we want hash functions to be as fast as possible, as long as they can uphold their security guarantees.
A maybe-interesting sidebar: If your job is to design a password hash, you still want to take a fast hash function as a starting point. All the effort that goes into making the hash function both fast and secure gives you confidence that an attacker isn't going to be able to find some shortcut that lets them compute the hash it more quickly than everyone else.
Hashes must only be slow in certain circumstances where the plaintext of the hash is vulnerable to a "pre-image" attack. This means that the entropy of the plaintext is very low and there are very few possibilities. In these cases you use extremely slow hashes like bcrypt or be forced to use AES encryption before hashing. For example: Passwords, Credit Card Security Digits, Names, etc...
Plaintext that has a LOT of entropy does not need a slow hash function. You can usually get by with SHA-256. For example: digital signatures, AES keys, Public/Private keys, large file hashes, etc...
In addition, if you are only trying to verify integrity with no need for security, a quick hash function is useful. Like a checksum on a file system. In this case a fast hash function is more important. EG: https://ext4.wiki.kernel.org/index.php/Ext4_Metadata_Checksu...
All of this is mostly right, but I have some minor corrections:
> or be forced to use AES encryption before hashing
Encrypting before hashing doesn't actually help us, if our hash input has low entropy. The problem is, what AES key are we going to use? If the key is public/constant, the attacker can just repeat the AES step themselves while they try to guess the input. On the other hand, if the key is secret, then we need to store it somewhere. Most likely, we're going to store it in the same place/database where we store the hash. (In this case it's basically the same as a random "salt".) But that means than at attacker who steals password hashes is probably going to steal salts at the same time.
> large file hashes
This is often true, but we have to be careful. What we really care about isn't the size of the file per se, but the number of possible inputs the attacker has to guess. So for example, if we hash a video file we downloaded from a bittorrent site, it might be easy for someone else to figure out what file that was, because even though the file is large, the total number of files on these sites isn't so large.
You are entirely correct. Unfortunately anything short of a textbook is going to be "Mostly Right" when it comes to security/encryption.
> Encrypting before hashing doesn't actually help us
Technically correct. There is no additional need to hash once you are encrypting already. However, when compliance tells you that you need to use hashing but you still need sub 10ms performance you sometimes need to use encryption. The disadvantage is the necessity for key management, HSMs, etc... Anything with low enough entropy is going to NEED encryption unless you are using ridiculous BCrypt or PBKDF2 parameters and are willing to wait a day or two to verify a hash.
> Large file hashes
If you salt the file data appropriately this is not an issue. The salt can be a part of the encrypted file ensuring that your hash doesn't collide.
PoW systems rely on the "phone a friend method" as well. When you download a Bitcoin client from a "friend", you are trusting them to honestly introduce you to the network. If you fall asleep for a period of years, you have to trust your friends to honestly inform you of all of the PoW forks and policy changes that have occurred over that interval. The only difference is that PoS blockchain clients must be bundled with a modestly-recent block hash along with the thousands of lines of code that you have no practical way to audit.
The problem eventually reduces to Ken Thompson's "Trusting Trust" [1] problem. There's no way to externally validate the honesty of any system (cryptocurrency, or otherwise).
You really don't need to trust a "friend" while bootstrapping into the network with PoW, because the proof of work is irrevocably embedded within the blockchain, and the real world cost of creating those blocks can be pretty easily estimated.
So long as you have a general idea of how much hash power is being used currently for the network, or even just how efficient ASIC computing is in general at your point in history, you can work out how great the hashing difficulty should be. You can trivially verify that the block hash with a large number of preceding zeros, e.g. 0000000000000000000b98dd8e7504793c0644cb0c27eb98f06aab9ea93c4ec2, is the hash of block it's attached to, and that a hash value that small would require a huge amount of energy to find. And every block beneath it also required a huge amount of energy, creating a huge real world economic cost to produce. You can't fake that chain without equivalent sacrifice of energy and compute resources.
Anyone trying to deceive you with a false chain would have to expend approximately as much energy as the entire legitimate bitcoin network does, and then keep doing it for as long as they want to deceive you. Sure, that theoretically could happen, but the economic incentives to do it just aren't there.
It seems that PoW does not need phone a friend to compare "which of these two chains is the true one", whilst PoS does need phone a friend for that.
However, that presumes all forks are soft forks; that you are presented a correct chain; that you want the soft fork with consensus rules accepted by most miners. (If verifying with an old bitcoin client the BCH BCT split will be resolved for you without you having a say.
In summary, PoW has less need for Phone a Friend than PoS. But it still has some problems.
How the hell do you know? You've just admitted that you don't actually know how PoS and PoW work. You've repeatedly refused to "do your homework" by researching what's known about these things. And yet you have repeatedly been rude to other people who have done their homework, and have informed opinions, unlike you. Just shut up and stop talking about blockchains. You're an entitled internet nobody.
I have regrets about calling you a "nobody". I was annoyed, but that's going too far, and I apologise for saying that. Almost no one deserves that level of vitriol, especially if at worst they're just being annoying. And I think I get annoyed too quickly.
Indeed. And even if you posit a PoW currency which never has policy changes, unlike Bitcoin or any other major cryptocurrency…
And you assume that attackers will never have enough computing resources to execute a 51% attack – which could happen because the currency’s value falls enough that people stop mining it, because an extraordinarily well-funded entity decides to attack it, or because someone manages to hack the miners…
Then you do gain the security guarantee that if you see multiple competing branches of the blockchain, you’ll know which branch is the correct one (namely, whichever is longest). However, you’re still relying on phoning your “friends” (nodes you’re aware of) to tell you what blocks exist! If they all keep the true longest branch a secret from you (or, say, someone blocks your Internet connection to the nodes that aren’t willing to do so), then you will think the next longest branch is the correct one.
To be fair, that isn’t the most practical attack. But none of the risks being discussed here are remotely practical. In practice, nobody wants to connect an outdated client to a blockchain network because it risks (a) getting yourself exploited through known vulnerabilities in the client, (b) not working due to backwards incompatible protocol changes or bugs, or (c) missing a hard fork that might have happened over disagreements in policy changes (because there are always policy changes). So you update your client, and that means you have to rely on a “friend” to tell you which software you should be running.
> But it's a threat for single nodes not for the network as a whole.
Indeed, but the same is true for attacks on "weak subjectivity" proof-of-stake. They're only a threat for nodes that have been disconnected for a long time (months) before they try to reconnect.
Except for the part where eclipse attacks can be resolved by simply feeding my node more data (it's not a problem if some of it is lies), while "weak subjectivity" requires recourse to an external authority.
i don't know as much about this as you, but it seems to me that the attack you describe in the blog post would also require a successful eclipse attack?
My understanding is that the attack you describe involves a cabal of "evil" validators signing some alternate chain (call it the "fake" chain) long after their stake is withdrawn, creating a fork in the distant past. Before they did this, they pretended to be good validators, which meant they signed the "real" chain's blocks and then signed the withdraw transaction. So after the attack, there are two conflicting sets of signatures signed using the evil cabal's private keys; those on the fake chain, and those on the real chain. So anyone in possession of both of these sets of signatures can conclude that the validators in the cabal are "evil", and then they can see that once the cabal's support is removed from consideration, the real chain had more valid validator support (at the time of the fork, in the distant past). If this line of reasoning is correct, that suggests that anyone who is aware of both sets of signatures can identify the real chain?
> So after the attack, there are two conflicting sets of signatures signed using the evil cabal's private keys; those on the fake chain, and those on the real chain. So anyone in possession of both of these sets of signatures can conclude that the validators in the cabal are "evil", and then they can see that once the cabal's support is removed from consideration, the real chain had more valid validator support (at the time of the fork, in the distant past).
I think this is where you get the problem - if you just have two sets of signatures, how do you tell which is legitimate and which one isn't? How do you conclude in which set the cabal was lying?
An eclipse attack is so named because it requires you to keep all the light out so they're kept in the dark. But here, since there's no internal mechanism to tell the two chains apart, you don't only need the accurate information, but also outside information about which one is accurate.
> I think this is where you get the problem - if you just have two sets of signatures, how do you tell which is legitimate and which one isn't? How do you conclude in which set the cabal was lying?
I feel like you should be able to deduce it from the distribution of participation after the fork, right?
The “fake” chain would lose all honest verifiers (and all transactions from honest wallets?) which seems like it would be pretty detectable with simple statistical analysis. Staked nodes not participating (and active wallets not transacting) becomes less and less likely the longer the post-fork chain is.
> The “fake” chain would lose all honest verifiers (and all transactions from honest wallets?) which seems like it would be pretty detectable with simple statistical analysis. Staked nodes not participating (and active wallets not transacting) becomes less and less likely the longer the post-fork chain is.
But you don't know who's honest - you may as well be saying the real chain lost all the dishonest verifiers.
Exactly - that's where statistical analysis (like fakespot) comes in.
For each chain you'd be able to look at the age, stake & historical participation level of the post-fork participants and get a pretty good idea which (if either) of the chains is real. The absence of honest participants should look a lot different than the absence of dishonest ones.
Granted, this method is not nearly as simple as checking the number of 0s on a hash, but I would imagine it to be quite difficult to circumvent.
You any specific verifier/wallet, you won't - but the fake chain will have 0 uncompromised actors after the fork.
Which means that large stakeholders suddenly stop verifying blocks. Long-term active wallets stop transacting.
The same might be true for both chains after the fork, but I would imagine the fake one would have a larger change in participation (weighting older wallets and larger stakes) than the real one.
If you actually see two conflicting chains (either proof-of-work or proof-of-stake) with large numbers of people vouching for both, then the correct chain is not necessarily "whichever one is longer". Well, it could be, for you; "correct" is subjective. But by assumption in this scenario, a large number of people disagree, and you might want to transact with some of them. There is no way for software to decide this objectively; it has to ask the user to decide based on factors external to the network.
Where proof-of-work really does have an advantage is that you can more easily distinguish that scenario from the scenario where either one of the chains is actually a Sybil attack, i.e. a single attacker pretending to be a large number of people. Similarly, if you only see a single chain, with proof-of-work you can try to detect an eclipse attack (which implies a Sybil attack) by seeing if the hashrate has gone down dramatically.
That's a real advantage. I don't think it's even close to enough to mitigate proof of work's disadvantages, especially since the circumstances where it would practically come into play are extremely unlikely, but it's not nothing.
However, it's undermined by the fact that proof of work naturally encourages centralization. Bitcoin is centralized enough that it's not completely impossible for the vast majority of the hashrate to end up on one side of a fork (either soft or hard), while the vast majority of users and developers end up on the other side. (To be clear, this is very, very unlikely to actually happen, but so are all of the attacks we're talking about.) If this happens, the objective proof-of-work standard will side with the miners, but not with the people you actually want to transact with.
Of course, a proof of stake currency can also suffer a schism, but there is (probably) less tendency for stakers to be centralized, and if a schism did occur, at least the client wouldn't provide a false sense of objectivity.
None of this actually refutes my point. You're just suggesting that the fact that PoS is incapable of producing a consensus is outweighed by it allegedly being more decentralized. Be that as it may, it's not relevant to this thread.
Is it even true? Steemit had the exchanges do a hostile takeover, because everyone was staking through them.
I think the difference is which kind of hash you needed.
For PoW, you'd have to know the hash of the start of the chain (the "genesis block") in advance to verify you downloaded the correct chain. That's true, but this hash doesn't change during operation. You could get that hash from a history book if you will.
For PoS, the hash is from the end of the chain and therefore constantly changing. This means the challenge of finding out whether the hash is the right one is a lot more real than in the PoW case, because there is no "common knowledge" to go by which hash is right.
> For PoW, you'd have to know the hash of the start of the chain (the "genesis block") in advance to verify you downloaded the correct chain.
Nope. You could fork the chain at a period of low difficulty and it would still stem from the genesis block. It would either be a short chain, or have clearly low difficulty though, so it wouldnt fool anyone knowledgeable. Im not sure how you would leverage that chain for fraud.
A while ago bitcoin clients changed from facoring the 'longest' chain to favoring the chain with the most work done on it. (To prevent long chains with low difficulty)
The client can choose properly, but it needs to "call a friend" in order to get the options - if the client doesn't receive the proper chain but only fake ones, it will chose the fake one with the most work done on it.
You need to fork at low difficulty if you want to significantly lengthen the chain from that point, because creating a high difficulty, long chain that is valid is hard.
But-- there's nothing to preclude you making big steps up in difficulty at the end of the chain. It means that one evaluating the length of the chain for authenticity really needs to integrate the difficulty over the entire chain and not just look at the number of blocks.
Suppose I'm a new node and want to verify the blockchain. How do I verify that each block was mined with the correct difficulty?
I'd need some record about the actual real-world timestamps for each block. Then I could say something like "duration between block x and block x+1 was > 10 min, so the down-adjustment in block x+5 is justified".
But if those timestamps were stored on-chain, an attacker could simply lie about them and keep difficulty artificially low on its alternative chain.
On the other hand, if we had some un-forgeable record of block timestamps, wouldn't this solve the double-spend problem all on its own? Would we even need PoW at this point?
> For PoW, you'd have to know the hash of the start of the chain (the "genesis block") in advance to verify you downloaded the correct chain.
No. For Bitcoin you can accept a chain with an arbitrary starting point and you would still arrive at the same chain everyone else uses.
Although you do need to have an idea of the earliest acceptable starting point-in-time — e.g. verifying a low-difficulty chain starting the year 200,000 BC (with one block every 10 minutes) would take quite a while
Because of withdrawal delays, the PoS hash isn't from the end of the chain, but from a few months before. So it changes only about as often as client software updates.
Finally someone actually mentioning the code. In PoS "trust" must exist along several points in time before you can engage with the system - and the most notable point being trusting that the rules (written in the code) are of your desire.
With PoW you don't care about the software code. The rules are dominated by the PoW because it literally proves to you which is the chain where most people are interested in, because literally no single entity could burn that much electricity.
With PoS on the other hand you kind of need these checkpoints in the actual software and then you have to activate this entire new trust model where you have to trust the client code, and where it came from etc. I could literally come up with an entire fake chain on my computer and present it to you and without client-checkpoints there would be no way for you to not accept my chain compared to your current one.
With PoW I don't have to trust anything. If the majority next year decides to change the rules, so be it. The majority has spoken.
I was talking about the consensus part. You don't need any client code to understand which is the agreed-upon chain, you verify the hash was generated using lots of energy.
For transacting indeed you need to trust the various clients, but that's easy and can be done once. With the consensus isn't being tampered with, and, more importantly that others are using other types of rules.
Is the threat of long range attacks in PoS any worse than PoW in practice?
Bitcoin for example still relies on a list of hardcoded nodes for bootstrapping clients. Not to mention very few people actually bother to verify the full chain (360GB and counting) from genesis.
As for auditing the the integrity of the code or binary, it is signed by GPG keys hosted on public key servers accessed using X509 certificates pinned by a a couple of trust anchors preloaded in your OS. So much for distributed consensus...
> As for auditing the the integrity of the code or binary, it is signed by GPG keys > hosted on public key servers accessed using X509 certificates pinned by a a
> couple of trust anchors preloaded in your OS. So much for distributed consensus...
You can literally validate the entire chain with a simple python script. Millions of those on github.
>Not to mention very few people actually bother to verify the full chain (360GB and counting) from genesis.
Absolutely wrong. The chain is validated in its entirety upon first sync. 100% from genesis to tip.
>Bitcoin for example still relies on a list of hardcoded nodes for bootstrapping clients.
It doesn't. Longest valid chain with most work is the canonical chain. Hardcoded seed nodes exist to speed up the discovery.
> You can literally validate the entire chain with a simple python script.
I challenge you to present a "simple python script" that implements the exact bitcoin consensus rules (as codified in bitcoin core). Bitcoin is not all that simple and there's a nontrivial amount of complexity in bitcoin script alone [1].
> The chain is validated in its entirety upon first sync. 100% from genesis to tip.
The default behavior is to skip signature verification for all signatures before some relatively recent block [2].
You're misunderstanding the default behavior which is fine becaue it's commonly misunderstood and discussed. At any rate signature verification is not skipped by default, what assumevalid skips is script verification. Everything else including UXTO, proof of work, the transactions themselves, are validated.
The Bitcoin Core client includes a hardcoded list of DNS servers that point to thousands of nodes. These lists get updated frequently by different people. Other clients may use other lists. What is the threat model you're suggesting here, exactly? Do you know any other way to bootstrap a peer to peer network without centralised authorities?
All network participants are forced to verify the full chain from genesis. Some might be OK with validating block header signatures only, and not the full transaction set. It's a tradeoff.
You don't need to use those public key servers if you somehow distrust the CA certificates in your OS. Feel free to contact the repository maintainers or whatever else floats your boat.
Anyway, bitcoin is an open source protocol, not a particular client implementation. If you distrust everything and everyone, no one can stop you from building your own client that works with the rest of the network.
> Do you know any other way to bootstrap a peer to peer network without centralised authorities?
I’m not the parent, but – no, I don’t. But that’s exactly the point. The need to bootstrap from centralized authorities is what’s supposedly so bad about weak subjectivity in proof-of-stake. Yet in practice, it’s needed with proof-of-work as well.
Maybe I didn't word it right, but I wasn't calling bitcoin's method reliant on centralised authorities, just asking if there were more methods out there that weren't as well.
Bitcoin is an open source permissionless protocol, so you have multiple clients to chose from, each with their own list of bootstrapping nodes, many open source where you can submit a PR to add your node too. You can even build your own client and point to whatever you want. You can also just ignore them and just point directly to nodes in a list from a public forum, a private chat, whatever.
Also, you're not just connected to those bootstrapping nodes: you use them to find the rest of the peers in the network.
It seems that PoW is like recursion. You don't get it at all until you completely get it. It's a leap to understand it and somehow many very technical people don't understand a seemingly simple protocol even a decade after it became mainstream and is threatening national banks due to the very characteristics these people claim it doesn't have.
> Bitcoin is an open source permissionless protocol, so you have multiple clients to chose from, each with their own list of bootstrapping nodes, many open source where you can submit a PR to add your node too. You can even build your own client and point to whatever you want. You can also just ignore them and just point directly to nodes in a list from a public forum, a private chat, whatever.
I characterized this as relying on centralized authorities (albeit several of them), but sure, it can also be considered decentralized to some extent.
The point is that it's a mechanism outside of the proof-of-work network itself. Instead of relying on a machine to reach consensus via a formal protocol, you the human are probing for a social consensus by evaluating statements made by other humans (via GitHub, public forums, or private chats, or just talking to people in person).
In both proof-of-work and proof-of-stake, you need to find social consensus in order to initially obtain the software, after which point you can rely on the network's consensus.
The difference with proof-of-stake is that you have to redo this if you disconnect from the network for months on end.
In practice, for a variety of reasons, practically all users of cryptocurrencies download regular software updates, and thus continue to rely on social consensus, regardless of whether the currency is proof-of-work or proof-of-stake.
I want to take a moment to note what you're doing here. You're making a negative argument, in want of a better word. It goes something like this:
1. X is a problem?
2. But Y is also a problem, in my opinion.
3. X and Y are both the same, I think.
4. Therefore X is not a problem.
We can - theoretically - verify the correctness of PoW software by downloading the source code, reading it over, etc. We can also refuse to update, reducing ourselves to SPV security. We can internally verify the checkpoints using 100% objective standards. There are other things as well. This is not the case for PoS, where our "signature A existed at time B" has to be taken as faith, or evidence of things unseen. There is no internal way to verify the veracity of such a statement.
The fact that users aren't personally doing this, is not the same as saying it makes no difference whether they are able to or not. I'm not personally going to withdraw all the money in my bank account - that would be ridiculous - but if the bank informed me I was no longer able to withdraw the money in my account, that would not be suitable at all. The assurance that I can do it makes it so that I don't have to.
It seems like you reject this premise, maintaining that PoW networks are objectively verifiable? But you didn't really refute the parent's point there, which was that there are no "objective standards" in deciding which bootstrap nodes to use; it's ultimately a matter of trust. If I trust the wrong bootstrap nodes, I can be eclipsed from the real network.
Granted, I only have to trust that a single bootstrap node from the list will faithfully connect me to the honest network. But PoS involves a very similar 1-of-n trust model; I can request checkpoints from n semi-trusted sources and check that they match.
Also, granted, if I pick bad bootstrap nodes, I can still detect if I'm being eclipsed by looking at the hash rate. But how do I know what hash rate to expect? I could check n websites with hash rate charts, but that brings us back to 1-of-n trust.
> 4. Therefore X is not a problem.
IMO it's a manageable problem. Users just need to be cognisant of these trust assumptions they're relying on, and be thoughtful about picking semi-trusted peers (whether bootstrap nodes or checkpoint providers).
> It seems like you reject this premise, maintaining that PoW networks are objectively verifiable? But you didn't really refute the parent's point there, which was that there are no "objective standards" in deciding which bootstrap nodes to use; it's ultimately a matter of trust. If I trust the wrong bootstrap nodes, I can be eclipsed from the real network.
Right, but it's not about trust in the same way. I can add an infinite list of bootstrap nodes. Quantity matters, not quality.
> But PoS involves a very similar 1-of-n trust model; I can request checkpoints from n semi-trusted sources and check that they match.
"Very similar," not the same. You need "semi-trusted sources", and there's no objective standard in case they disagree.
Hi Nick, very well said and this is precisely my point as well.
Satoshi tried to convince us that we could decentralise trust by doing honest work instead of relying on authority. It turns out that doing work is actually pretty hard, people are lazy, and security is still the nemesis of efficiency.
> Do you know any other way to bootstrap a peer to peer network without centralised authorities?
In IPv4 a client might have a chance at auto-discovering peers.
It's also not necessary to rely on a single centralized authority. There are many things (DNS, Encyclopedias, Linux kernel mirrors, etc.) where the majority of existing centralized authorities agree with each other.
DNS is based around central authority though. Every root zone has name servers that serve as the authority. Their responses get cached at various levels but take those servers offline until TTLs start expiring and everything breaks.
What part of DNS do you feel is possible without a centralized authority?
> Bitcoin for example still relies on a list of hardcoded nodes for bootstrapping clients.
It does, but it doesn't have to. You can use any mechanism you want to obtain one initial node and take it from there. You will still be connected to the network just as well, and you will be guaranteed to obtain the same results. This differs from Proof of Stake, where the quality of the results will be influenced by the quality of the bootstrap.
I verified the full chain a couple of weeks ago (But I admit I trusted umbrel to choose the "correct" bitcoin-core software to run), It took less than 3 days to sync on a Rpi4
'Policy changes' and hard forks have about as much to do with PoW as whether the Federal authorities should ban cryptocurrencies or not - they're outside the realm of consensus algorithms. In PoW there are no friends. If your blockchain is incorrect (i.e not the longest) your transactions on it are invalid and will be rejected by the rest of the network.
> If your blockchain is incorrect (i.e not the longest) your transactions on it are invalid
If your chain tip is on the dead side of a hard fork (i.e. if the majority of the network will predictably soon finish switching away from software which considers your chain tip valid, to software which considers your chain tip invalid), then nobody cares if your chain tip is the longest in the interrim, or how long you still hold out running the software that considers your chain tip valid. Your side of the fork no longer holds any economic value as a platform for transactions, so nobody will participate in it. You'll just be out there mining blocks all alone, blocks that say you earn all the virtual tokens, but where those tokens are worthless on your side of the fork.
It's a bit like how, in old pre *serv IRC networks, in cases of netsplits, you could end up on a partition of the network where you were the only one in a previously-moderated channel; and so you could effectively do whatever you wanted in that channel. But it didn't really matter, because nobody could hear you.
Um, yes. I should have phrased that as 'your transactions based on it are invalid in the network'. You just described consensus working correctly, but like I said hard-forks and policy changes are outside the scope of PoW, so saying PoW does not handle hard-forks is not really a valid criticism of PoW.
"nobody cares if your chain tip is the longest in the interrim,"
Except the people you bought something real-world from, once they figure out that their "tipcoin" is worthless. So now it's a question of convincing some people that your technobabble is valid enough. How hard is that?
No, policy changes makes for a new blockchain. That's what usually referred to as a "hard fork", as opposed to a "soft fork" where consensus rules are only allowed to get stricter, exactly beacuse ownership of a coin should be guaranteed forever.
You could follow the consensus rules set out from the beginning and you would still end up on today's majority chain.
I believe there were a couple of early bug fixes along the way, which makes this not strictly true. As in the original first release of the software not actually capable of downloading all of the chain, which some people love to point to as a proof of it being a fallible system. This is probably true but doesn't really detract from the original point of guaranteed ownership by never relaxing the consensus rules.
not really, if you fall asleep for a period of years, you can still get a signal of how genuine any proposed fork is by observing the chain of blocks and their difficulties. that's the crucial bit of any PoW system - you can't fake the energy that was spent producing the chain. that's a way to externally validate the honesty of a system and a major scientific breakthrough that satoshi discovered.
The difference in Proof of Stake is a lawsuit could force the distributor of the software to change the hash to one where coins weren’t stolen. As most developers are not pseudonymous, this poses a threat to the honesty of the system.
You mention “POW forks”, but Bitcoin’s POW has never been hard forked: you’d need to trust a Bitcoin expert to tell you if it was a good idea.
> The difference in Proof of Stake is a lawsuit could force the distributor of the software to change the hash to one where coins weren’t stolen.
And with proof of work a lawsuit could force the distributor to change the consensus rule so that a particular transaction is invalid - just as Ethereum did voluntarily with the original DAO.
> You mention “POW forks”, but Bitcoin’s POW has never been hard forked
Instead it’s been soft forked, which turns the consensus rules into a popularity contest. If a soft fork produces two competing branches of the blockchain, old clients will go with whichever branch has more mining power. Which means you open yourself up to interesting attacks like convincing 51% to literally steal the funds of the other 49% (which is much worse than a mere double spend). Or, more realistically, in the case of a contentious soft fork that ends up roughly fifty-fifty, you could ‘just’ end up on a different side of the fork from the people you want to transact with. Either way, soft forks don’t make the downsides of policy changes go away.
Changing consensus rules requires coordinating a fork. This requires coordinating developers, miners and node operators. That may fly in pseudo decentralised chains where the community accepts whatever the leader says so, and even so, at high risk. In bitcoin, for instance, where there is no leader, this would never be a viable scenario.
Soft forks don't force you to download and run new clients just to be able to use the network, which is an important difference. You can use your existing client, you just don't have the new features and don't run validations on them.
The greatest risk on soft forks is that chain split you mention. That's why any reasonable soft fork deployment requires a long time window with a large majority of hashrate signaling support (like 95%).
Changing a PoS checkpoint would also require coordinating a fork. Even if a dev team were forced to make the change, they couldn't make everyone go along with it.
> The difference in Proof of Stake is a lawsuit could force the distributor of the software to change the hash to one where coins weren’t stolen.
In Proof of Work, a lawsuit could force the distributor of the software to hard-code a transaction that reverses the coin theft. But in both the PoS case and the PoW case, anyone using that client would be partitioned off from the honest network majority.
> You mention “POW forks”, but Bitcoin’s POW has never been hard forked: you’d need to trust a Bitcoin expert to tell you if it was a good idea.
Bitcoin's PoW forked in 2013, when a database upgrade to the software made it incompatible between two recent versions. The Bitcoin developers had to jump in and tell people which PoW fork to follow and which one to abandon.
It would dilute the messages in the guidance, it was also be quite patronising for anyone who has just been through a lengthy decision making process and concluded they still need a VPN, to be told they shouldn't be using a VPN.
The client sends the request headers (including the x-amz-content-sha256 header) to the signer, and the signer responds with a valid S3 PUT request (minus body). The client takes the signer's response, appends its chosen request payload, and uploads it to S3. With such a system, you can implement a signer in a lambda function, and the lambda function enforces the content-addressed invariant.
Unfortunately it doesn't work natively with multipart: while SigV4+S3 enables you to enforce the SHA256 of each individual part, you can't enforce the SHA256 of the entire object. If you really want, you can invent your own tree hashing format atop SHA256, and enforce content-addressability on that.
I have a blog post [1] that goes into more depth on signers in general.
[1] https://josnyder.com/blog/2024/patterns_in_s3_data_access.ht...