As Dan notes, a lot of software is just...not very good. It either isn't upfront with flaws (as in the case of the Postgres -> Snowflake tool), has too much scope, or is abstracted poorly. Finding things to buy/use (as in the case of open source) can often eat a lot more time than you anticipate.

I've been dipping my toes into the JS ecosystem, and I keep bumping into the fact that using mentally cheap signals of quality (such as stars or DL counts) almost never indicates the quality of the thing itself. Winners seem to be randomly chosen, almost! The only way to assess is to read the code and try integrating it in.

I'd go farther to argue that the larger an ecosystem/market is, the more untrustworthy it behaves as a whole, simply due to the size, and the types of people attracted to it who want to get influence/money. See also: appliances that everyone needs.

Unfortunately, most consumers, recruiters and sometimes hiring managers are in a position of information assymetry vis a vis the people selling them something. That is, consumers rely on the self-reporting of vendors which purport to be experts.

https://vonnik.substack.com/p/the-expert-layman-problem

In contrast, product information invites people crafting lies for an advantage.

Consumers rely on advertised claims being truthful.

It is not a matter of people badly predicting their own needs in most cases,though there are some that do have problems with this. It is a matter of people being misled by false information and trying to course correct after that information comes to light.

In a world where lies of omission and ambiguity towards borderline malice isn't considered an outright lie, but the sales reps do make those outright lies, and fake reviewers are not punished; there are real problems especially when the presumption is they aren't doing this (when in many markets this is exactly the standard behavior, and even academic studies show these things happen regularly).

Presumptions are just assumptions. Someone will take advantage of the grey unenforced area to push a product that may not be as professionally tested as they claim (or even finished). I've certainly run into a lot of these bait and switch types in my long professional career. The general term to describe this is snake-oil, and with the concentration of the market over time (increasing marketshare less participants), this only gets worse.

Starving for talent my ass. His portfolio companies have infinite appetite for talent at zero cost, the minute one person wants one point of the upside they’re right back to starving for free talent.

Silicon Valley is the ultimate thought experiment in how wealth inequality plays out when resources are effectively inexhaustible. You don’t have to speculate about a post-scarcity world, this is a post-scarcity world. Ballers in SV ship a billion in revenue on a Tuesday. And yet somehow it’s Andreessen who ends up with all the chips at the end of the night.

These talking heads don’t have a plan, they don’t know what their next big payday will be, they don’t code, they don’t design, they don’t sell anything other than their own personal brand, they don’t add value.

They’re just patient and connected like zen spiders sitting on a web: they’ll learn first about a new big thing, they’ll be there immediately, their friends will wire up the deal in their favor, and they’ll do a TED talk a few years later about how making themselves absurdly wealthy with no real effort is somehow the future of humanity.

They openly advertise their glee at the (ridiculous) idea that soon some autoregressive language model will do all the work and the owners of NIDIA cards can just pocket it all.

In the 90s there was this meme of a yuppy couple doing a business from their couch via “The Information Superhighway”: outsource everything, all you need is a laptop, a glass of Chardonnay, and a lot of cheek.

pmarca should spend less time yakking about AI on Lex stream and more time learning AI on geohotz stream.

It would be different if it’s someone e.g. very high up at a F500 selling something, even with a huge information asymmetry, because it’s still possible to bank on their credibility. (Assuming they offer sufficiently many guarantees signed by sufficiently many people.)

I could think of reasons both in favor and against, but I'm curious about your rationale.

I wish morep people understood the "Kardashian effect" as I like to call it -- the most popular thing is only most popular because it was already popular. I think in almost everything in my life and in every domain, #2 or #3 is better-suited (for my preferences and needs).

A year or two ago on HN I read a short blog post about omitting the word "best" from internet searches and being more specific in your criteria (e.g. "car with best resale value" instead of "best car"), and it has made my life and way of thinking a lot better

I like to explore alternatives to the most popular choice, but more often than not I end up back at the #1 consensus choice.

I have some friends who simply must pick the #2 or #3 choice in every domain. They always have an elaborate justification for why it’s better. From my point of view it seems driven by contrarianism.

Some times they pick some interesting alternatives that I explore. Most of the time they end up with also-ran purchases that die off. I joke that my one friend is the best predictor of impending product line cancellation that I know. He used a Zune when everyone went iPod. He went Windows Phone when iPhone and Android were front runners. He event eschewed Instagram for some other platform that he was sure was going to win the social media wars, but was actually so unnoteworthy that I can’t even remember the name right now.

  the same group of consumers has an outsized tendency to purchase all kinds of failed products, time after time, flop after flop

  In a key part of the study, the researchers studied consumers whose purchases flop at least 50 percent of the time, and saw pronounced effects when these harbingers of failure buy products. When the percentage of total sales of a product accounted for by these consumers increases from 25 to 50 percent, the probability of success for that product decreases by 31 percent. And when the harbingers buy a product at least three times, it’s really bad news: The probability of success for that product drops 56 percent

Always a great cue to instantly stop considering a product.

Perhaps I should set up a Patreon where people can give me money to not watch things.

(Admittedly, this may be a mathematical artifact. Cancelled shows have less opportunity to decline in quality. At one point, The Dragon Prince was my all-time favorite show. I haven't even bothered to watch the last several seasons. It is even possible that the people cancelling the show are able to accurately predict that something is going to go downhill in the future, though I doubt it.)

I got the vectrex too, and an Atari STM I think, it wasn't quite the same as an Atari STFM. When I wanted to learn piano/keyboard, he bought me a frickin keytar, a Yamaha SHS-10, instead of lessons or a simple full size like I asked for.

Instead of a gift certificate to get some clothes, he got me a gimmicky Canon SLR that ate batteries and that I couldn't afford to develop the film.

He was a very strange person. Sometimes incredibly funny and generous, other times hateful and selfish.

Edited to add: sorry for the trauma dump. I have no idea what point, if any, that I was originally trying to make.

I would have liked to have eaten something other than boiled potatoes and peas in my birthday. I would have liked to go to the ice rink with my friends. Instead I got a camera I couldn't afford to use.

Since that can often mean extra effort/support, I won't recommend such things to others. I'll try to pick something that will be the least trouble for them.

Not to say that every #1 popular item is always the best, but definitely a lot more than never.

Popularity is only a decent proxy for consensus if people actually look at the other options. I've been burned by trusting this metric more times than I can count.

I may be one of these types, but at least in many of my cases, I don't really know that it mattered in the end?

Maybe after a review I pick something that didn't win in the long term or even eventually exited the market because it wasn't popular enough. But, my requirements are almost never strictly that it's popular. What I end up with does typically do the job very well for the time I have it, and after few years the requirements may change or the need may go away completely. If one of my requirements is that a device is built with metal instead of plastic, maybe I never have to replace it.

Another example: Your friend had a Zune, but then I'm guessing they moved on to a phone, [because phones eventually] became better music players. If the Zune did all the things they wanted while they had it, especially if they had a unique need, maybe were happy with it. (Although, that isn't necessarily always the case.)

This doesn't seem quite as applicable for selecting software, though. Popularity often is part of what I look at there, because I want to know dependencies won't need replacing and support will be available. Additionally, you can potentially work with the developers so the selection iterates and grows into your requirements.

https://news.mit.edu/2015/harbinger-failure-consumers-unpopu...

Depending on the domain, there's a good chance #1 may also be a sponsored result.

Alternatively, the Mickey Mouse effect.

- ZFS. There are other filesystems that provide checksumming, CoW, etc. but ZFS is a proven solution in this space. I am happy work on BRTFS and bcache continues but they are not yet there.

- Debian. There are distros that do things better and differently but nobody has quite as smooth an experience, hardware support, security, and software availability all in one package. Alternatives are really good so it really says something about Debian that it is as popular as it is.

- ssh. There are some alternatives but when was the last time you heard an argument made in their favor?

I could go on but hopefully I’ve made my point. Sometimes the standard choice is the right now. And yes I know me picking Debian as a good distro can start a flame war :)

I often wondered how to strike the balance right on these things, since apparently all options can lead to success.

I think even he is a little averse to straight up saying it out loud (we’ve discussed it many times): software and product outcomes have gone to shit because we don’t enforce anti-trust law already on the books, let alone update it in light of 50 Moore doublings.

You want a smart phone? Here are two vendors with the same App Store vig at payday loan shark rates. You want to rent cloud compute? Here are several vendors where the 4th place vendor charges the same as the first place vendor. You build disruptive encrypted messaging app? You better live on a boat and watch your six like Jason Bourne Mr. Marlinspike.

Is it even a little surprising that everything from Google search to Netflix is a shittier version of itself ten years ago? There’s no incentive to make it compelling!

You have to flip the percentage. The fraction of money that goes to the intended destination after a 30% tax is 70%. The fraction of money that goes to the intended destination with a one year 100% APR loan is 50%.

Though there's no reason to assume a year in particular. If you take a six month loan at 100% APR, you have to pay back 141%. Paying back 141% is equivalent to a 29% tax. And if the best comparison is "six months of compounding payday loans" that's even worse than the initial comment suggested.

The better analogy is Tony Soprano with a migraine.

Their hardware sales shouldn't entitle them to a cut of the software used on it, and the hosting is not worth particularly much.

The value of their platform is in both hardware and software by the way.

Some internet platform thing stops being socially useful from competitive innovation and starts being an extractive rent?

The public has the power to dictate terms to the people running it. It’s a power the public hasn’t exercised a lot recently, but it’s only been about 30 years since LA 92, a little longer to Watts and Detroit.

It would be a grave error to mistake the public’s kindness for weakness.

If that's what you think matters above all else, would you support a literal mafia group taking a cut if they had started the market in that particular city?

> public property

It's a market, not property.

But Apple owns none of the iPhones, so why do they get a say?

> The value of their platform is in both hardware and software by the way.

They shouldn't be artificially tied together by DRM.

This is called being a "landlord" and it's actually completely legal.

You buy some land, you build a mall, you invite shopkeepers to set up shops and sell to customers visiting the mall, and the shopkeepers give you $$$$ every month, forever. If they ever stop paying, they lose their shops.

Even though it's shopkeepers that draw customers to the mall in the first place, and even though the shopkeepers are covering all the maintenance costs of the mall, you get paid anyway. Because you own the mall.

So if a mafia group created a market, took a cut from every shop, and threw out anyone who wouldn't pay them their cut? That's actually a legitimate business.

That analogy still doesn't quite hold as you legally agree to pay HOA dues when buying the property and agree the property can seized eventually if you don't pay. No such agreement is made between phone owners and Apple, Apple just sells you the thing and puts some of it's functionality behind ongoing fees.

I have heard the rent argument made for property tax and it holds better there. Own your house outright but fall behind on property taxes and the government can take your house and land - now that feels a lot more like rent.

There is no useful analogy from an idyllic high street with people shopping at leisure to what happens when you give sociopaths unbounded compute and legal carte blanche.

I struggle with this as a meme and I have a more and less charitable theory. My more charitable theory is that people never learned finance seriously. My less charitable theory is that people stand to benefit personally and are indifferent what it costs, as long as someone else pays the cost.

None of these concepts are in the bedrock law of the land: shopping mall, home owner’s association? Which amendment protects HOAs?

It is completely possible to change all of this, and while we should deliberate thoughtfully before making sweeping change, I mistrust anyone who just asserts dubious shit as bedrock laws of nature.

Maybe we’ve got a pretty sickening system composed of dubiously legal and strictly unethical status quos and it’s time to make some changes at arbitrary effort and cost.

Profit margins and low-friction competition go hand in hand, if one endorses the one without demanding the other?

I’ll show you a pretty small-minded person with distressingly little empathy.

Today that’s so captured that Mickey Mouse is still in copyright and insulin for diabetics costs more than the saline it’s in.

It defies both the law as written and the human sense that wrote it to advocate for unproductive rent seeking: sixteen years? With a few point patches along the way? Thirty percent.

The Europeans find it ridiculous, the FTC finds it ridiculous, anyone who likes innovation finds it ridiculous.

But it’s mostly ridiculous because history is unambiguous on this point: you tell the peasants to eat cake long enough and you’ll swing from a dockyard crane.

Let’s seek to avoid that kind of thing?

Why would that happen instead of Apple taking less profit overall?

I suspect that Apple's pricing for phones is based on primarily on what the customer is willing to pay rather than on the amount needed to be profitable.

And if they baked the price into the phones, at least that would be honest. But they likely wouldn't increase the phone price by that much.

If you didn't mean that, I'm not sure why you gave me that advice. I'll keep it in mind but I'd prefer we keep the discussion focused on what Apple is doing and whether it should be accepted by the general public, both in terms of popularity and legality. And Google too, because this discussion started about both app stores.

On human welfare? Every other day a new study comes out on the destructive force that heavily marketed smart phones represent: Phillip Morris and Enron put together couldn’t collapse the birth rate in a nation.

This debate is about economic activity as a good a priori. A lot of people assume that.

Some are ill-informed: they haven’t read the designer of GDP talk about the perils of GDP as a metric.

A small few know how mathematically comical the fucking Laffer Curve is and who thought it up and where and prey on the good intentions of the former group.

They’re going to catch a guillotine no matter how much caution I or anyone else advocates for.

Collapsing the birthrate?

Human welfare?

We're talking about the App Store my friend. You need to take it easy on the ideology.

It’s not done in a day but I’m on track to finish in 2025.

The problem with X is… the other people who use X.

We’re not in the “very large” regime?

> payday loan shark rates

Do you know what the retail markup on things you buy in your local shop?

If a product costs $1 from the producer, the distributer sells it to the retail shop at $2. And the retailer sells it at $4.

None of the examples of crappy software from the OP are Magnificent 7 companies, are they? Those companies seem to be capable of hiring the best talent.

A critical input into the build vs buy decision is whether your company is actually capable of building. Suppose Dan was advising the company that made the crappy Postgres/Snowflake sync software on whether to build vs buy a supporting component. Given that their main product is already crap, can they really be expected to build a better supporting component in-house? Seems like they might as well buy, and focus their energy on fixing the main product.

Dan advocates vertical integration here, and elsewhere says engineers get better compensation from FAANG than startups. This suggests that big companies made up of lots of talented engineers have natural efficiencies, relative to the alternatives.

A natural interpretation of "antitrust" is: break companies up into their smaller business units. That basically replaces an in-house "build" relationship with an inter-business "buy" relationship. So it seems antithetical to the OP.

Dan, if you're reading this, I encourage you to write a reply here on HN tearing me apart, like you did to all the other HN commenters.

I’ll challenge the notion that enriching mediocre people at the expense of society is either reasonable or useful. Enlightenment ideals around the sanctity of personal property fall apart when you extrapolate that to millions of people in passive index funds who don’t vote on governance.

Neither the United States Constitution nor the Universal Declaration of Human Rights nor the Geneva Conventions have any protections for rent seeking assholes as a bloc.

Don’t shoot the messenger: in my reading of history either one group backs down or the other slaughters them.

I’m lobbying for the peaceful resolution.

I’ll also advocate for a punitive wealth tax: not to raise revenue (who cares) but to cripple billionaires who aspire to nation state power. 50 million seems like plenty to live in arbitrary luxury but not enough to buy policy. Seize everything above that.

What happens after that is unclear. War possibly.

If legislature doesn't talk to industry before regulating, expect lots of incompetent, ignorant, economy-crippling regulation.

You don't have to change anything in the US if you want this. Just move to Germany.

>cripple billionaires

You might be able to persuade Trump of this, actually, if you sell it as increasing his power over the other billionaires.

More seriously -- It sounds like you have a strong desire to live in an authoritarian country, where the state possesses unchallenged primacy over its citizens. I support your right to do that. There are many available to choose from.

I'll bet if you do enough research, you can even identify an authoritarian country which also has crippling regulations. That should be perfect.

The United States was mounting a vigorous defense against the Warsaw Pact in 1960, and again in 1970, and straight up through 1979 before Reagan was elected and we took the Laffer Curve straight off the back of a napkin and to the legislature.

There is a way to run a participating democracy without a dystopian upwards wealth transfer, without unlimited political spending, without a catastrophic class segmentation that will lead to war. And we can do it all while opposing unbounded statism.

It sounds like you’re doing well under the current system and trying to frame it like that makes a system good.

Fuck you.

This will be my last reply in this thread (unless you apologize).

>These talking points went mainstream around the time Reagan got elected, around the time stuff like the Laffer curve got taken seriously.

Unfashionable isn't the same as wrong. It's telling that you're responding by saying "this is why you're unfashionable", rather than "this is why you're wrong".

>There is a way to run a participating democracy without a dystopian upwards wealth transfer, without unlimited political spending, without a catastrophic class segmentation that will lead to war. And we can do it all while opposing unbounded statism.

Are you aware that median wages in the US are some of the highest in the world?

https://ourworldindata.org/grapher/daily-median-income?tab=t...

Regarding class divisions and political spending, you might find these links interesting:

https://www.noahpinion.blog/p/america-doesnt-really-have-a-w...

https://slatestarcodex.com/2019/09/18/too-much-dark-money-in...

I actually care more about global inequality than inequality within the USA. I would sooner support effective foreign aid than welfare.

But really -- It's not your specific policy ideas that disturb me, so much as your attitude. Your politics aren't the politics of benevolence or charity. They're the politics of envy, ego, bitterness, and resentment. That never ends well:

https://iea.org.uk/publications/socialism-the-failed-idea-th...

>It sounds like you’re doing well under the current system and trying to frame it like that makes a system good.

Currently surviving off of savings due to a chronic illness. I haven't had a good source of income for multiple years. My Medicaid coverage just renewed. I rent a room in an undesirable area. I don't have a car. I won't get a chance to see my family for Christmas.

But you know what? I am "doing well". That's because "doing well" is more about your mentality than your material resources.

I hope you get the support you need in that area. Merry Christmas.

Sometimes I wonder if the free market has stopped working.

High-paying work in the US right now and software in particular is in a bit of a doldrums where the locally useful needles are like, friction to leave the platform, and the absolute limit of remnant ad load before people throw their phone at a wall, and taking a big cut out of everything, and lying constantly about what foundation models can and can’t do and shit.

I suspect it feels this way because this is a pretty exceptional run where the Valley just doesn’t have shit for the LPs. It was Web3 and then the Apple Watch (or vice versa) and then it was WeWork and then it was the Metaberse and now it’s “AI”.

But it will get better: dumb, lazy, corrupt management eventually gets rotated out. This crop is really hanging on by their fingernails, but the writing is on the wall.

>Sometimes I wonder if the free market has stopped working.

The "free market" means people have the freedom to compete. It doesn't mean they are guaranteed do so. You might as well blame society for getting lazy and complacent.

I really admire Google over time, it’s a special place that still has engineers you can’t find many places.

But once they let climber TPMs write their own performance review by just ransacking the dwell? An even worse crop (Altman, Amodei) was inevitable.

Without concrete examples your comment is just vibes.

You don't seem to be arguing that Google is a bad company actually. You're arguing that it was once a good/altruistic company, and now it's more ordinary. This strikes me as an entitled attitude. A company behaves in a good/altruistic way, people get used to it, they don't show appreciation, but they do start cursing if the altruism diminishes even a little. "No good deed goes unpunished"

The "free market" means nothing as nothing like that existed at any point of human history.

But “the effective freedom to compete” is a reasonable TLDR on versions of it that lead to peace and prosperity.

The “money for nothing” kind of capitalism? That kind gets regimes and their apologists killed in my reading of history.

I’ll admonish you to endorse a better plan.

If you think a company is doing a bad job of serving its customers, compete with it.

If you're right, and you can do a better job of serving those customers, you are likely to become rich.

If you aren't interested in competing, maybe you don't actually believe it is possible to do much better.

I agree that in principle, there can be a role for the government in removing structural barriers to competition. I haven't seen that role particularly well-articulated in this thread, though.

That, uh, is kind of what you expect in a competitive commodity market, isn't it? I don't think that's the example you want. (No disagreement about app stores).

When vendors 1-5 are all running 20, 30, sometimes 50 percent margins while their share varies from like 40% to like 5%?

That’s called price fixing.

The problem is that this disposition is not the norm for a technician, which is why I tend to prefer hiring and training engineers that are artistic, creative, and obsessive.

We incinerate the book every few years, but the fundamentals remain the same.

It would have been quicker and cheaper if the company just hired more actual developers to integrate properly with existing systems (and resulted in more featureful, less buggy applications), but the prospect of paying lower salaries for less qualified people to do the same end result (as promised by the slopware vendors) seems to be a siren song of sorts to management.

Winners are mostly chosen by effort put into marketing, not product quality.

The goal of the business it to make money, not make a great product.

They will make the product better if and only if it will lead to greater profits.

If you can make more money hiring more sales and marketing people to sell a broken product than hiring more engineers to fix the product, you're going to hire more sales and marketing people.

More broadly: products are the way to make profits, but they aren't the purpose of the venture.

I do not understand that at all, but maybe my reality is just substantially less mediated by "the conversation" that I don't get it. (Drizzle is fine BTW)

1. Many if not most "customers" (JS devs) don't understand how to evaluate quality deeply (thus, quality signals aren't trustworthy, making evaluation more difficult and resulting in further acceptance of using unevaluated products)

2. There's an overwhelming number of choices, with deprecation, replacement, and eclipsing all being fairly common - potentially even year-to-year

You kinda just have to try stuff out and work with what you've got.

Off the top of my head, the best software I use seems to fall into two categories:

- Closed source software that requires buying a license to use

- Open source software that is specifically made for developers and promises to do one job well

Whatever falls in the middle of those two categories tends to suffer, in my experience.

If you think about it, web based software tend not to fit neatly into either category. Most of them are the following:

- Closed source but are either too cheap or are free

- Open source but promises to do way too many things, and also too cheap or free (describes a lot of frameworks and design tools)

Web technology and JavaScript became the dumpster slut of software ecosystems. The end users are not given a big enough reason to pay for them adequately or at all, product owners care little about quality and reliability because it's way too easy to get a zillion low quality users to look at ads, and the barrier of entry for new JavaScript programmers is so low that it's full of people who never think philosophically about how code should be written.

I think an additional problem with the JavaScript ecosystem specifically is external resources are extremely easy to access and their cost is usually borne by end user resources. Therefore they're too tempting for many developers to avoid. Unfortunately the runtime environment of the end user rarely matches that of the developers and seemingly "cheap" resource access at development/test time isn't cheap for the end user.

JavaScript is happy to pull in some library hosted on some third party service at runtime. For the developer/tester this ends up cached by the browser and/or at the edge of the CDN. A developer may also have topologically close CDN endpoints. This inspires them to pull willy nilly from third party libraries because to them they're cheap to access and they save time writing some utility function.

The same goes for CSS, APIs, or media resources. With JavaScript the delivery is a client issue and costs can be pushed entirely onto the client. If pulling in an external resource(s) costs a developer non-trivial money to store and serve they'll put more effort into tree shaking or other delivery optimizations. They may omit it entirely.

I think this massively contributes to the LEGO piece construction of a lot of web apps. It also contributes to performance robbing things like tag managers that insert a bunch of incongruent plug-ins only at runtime from an unbounded number of sources.

And yes, you can find 10+ packages to do pretty much anything... of which 8 are abandoned and no longer work on up-to-date Node versions or depend on other packages with known vulnerabilities.

Then people started stacking more and more things on top of those libraries, creating a giant dependency morass of microdependencies that larger frameworks are now build on top of. And because all these frameworks do things just different enough from each other, every larger library that a dev would want to integrate with those frameworks now needs a specialized version to work with those frameworks.

In most languages, if you want to know how something works, you can usually dig into your dependency tree and by the time you hit the stdlib's optimization spaghetti/language internals, you'll probably have figured out what's causing your snag/what the library is expecting out of you. In JavaScript, you hit the dependency morass and have to give up. Most competent devs then decide to pick another language.

You can write very legible JavaScript these days, even without a framework, but it looks nothing like JavaScript used in a framework.

The other language I know of with this issue is, ironically, Rust.

The one thing salvaging Rust for now is a lack of similar frameworks, but who knows how long that will last.

Nobody else can understand how it's built, but for some reason that's not their problem? It's not like they're taking the time to understand how React is built, either. But as soon as you do something on your own, whoooa buddy. Cowboy programmer alert. It's not good engineering if it's a single, coherent, vertically integrated system. It's only good engineering if it's a mishmash of half-solutions teetering on top of each other.

I don't understand why people get so up in arms over npm modules, as if you could stand up code that does the same things in another language without having to manage dependencies.

I still do a lot of programming. And I expect my developers to be competent enough to read other people's code and figure out how it works, what it does, how to use it, based on the tests and plenty of extant examples.

I don't want developers who can only be productive in libraries that everyone else's is using/posting YouTube tutorials on/feeding LLM training corpus'.

The problem with adopting other people's software is that you have to make it work for your purposes, all while accepting it was only ever originally designed for their purposes. And if that's open source and you contribute to it, then you have to make sure all your changes don't break other people's work.

But with my own libraries, I can break anything I want. I have, like, 5 projects using them. It's not a big deal to discover an architectural problem and completely refactor it away to a newer, better design, propagating the change to all the others that use it in fairly short order.

And I don't have to argue with anyone about it. I can just do it and get the work done and prove it was the right thing to do long before any Github PR would ever get out of review.

For example, in the 2000s and 2010s, javascript had the lowest barrier to entry by far thanks to browser dev tools, so a lot of what you saw was people with no prior experience teaching people with no prior experience. These people are still around, and while plenty have improved, they still form the basis of modern javascript practice.

I also use almost no software that has been written by others. I use two or three external dependencies, in my work. Two PHP ones, and one Swift one. All are ones that I could write, myself, if they got hit by a bus, but they do a great job on it, and, as long as they remain bus-free, I'm happy to use their stuff.

The one exception is an app that I just released, using SwiftUI. I needed an admin tool that displays simple bar charts of app usage data. I was going to write my own UIKit bar chart widget, but SwiftUI has a fairly effective library, so I figured I'd use it, and see how SwiftUI is doing, irt shipping apps.

I think that I'll avoid using SwiftUI for a while longer. It's still not ready, but it has come a long way, since my first abortive attempts at using it. The app works, but I did some customization, like pinch-to-zoom, and that's where SwiftUI kicks you in the 'nads. As long as you stay in your lane, things are sick easy, but start driving on the shoulder, and you are in for some misery.

And that's the biggest problem with relying on someone else's code. They usually punish you for any "off-label" use. Apple has always been like that, but they usually let you get away with it. I go "off-label" all the time, because I don't want my apps to look like Apple Settings App panels. SwiftUI doesn't suffer deviance at all. Just adding pinch-to-zoom was a bit of a misery (but I got it going, after several days of banging my head, and it now works fine). Some frameworks and libraries won't let you deviate at all. You can't have any pudding, if you don't eat yer meat.

From there, I find two problems rather often. One, it doesn't actually do what it claims. Googling is fruitless because I must have been the first one to actually try it. Gotta love Googling an error and finding exactly one result - the source code. I sometimes open a bug report depending on how the project seems, and that gets anything from deaf ears to "oh thanks, we'll fix it in the April release" which is of little use to me now.

The other is something you touched on briefly, that the API or contract changes in unexpected ways over time. You can stave this off for a little while by pinning, but then you're missing out on bug fixes and new features. Which, especially for a middleware type thing, is usually a death sentence by bitrot.

You have none of these recourses with software. Yes, there are trial periods but that's not the same as buying and returning, and they're usually shorter than return windows.

And there's never any guarantee that it will "just work". We have managed to convince consumers to put up with software that "breaks" in a way that they would never do with hardware.

If the project appears to have several active contributors that would be good too.

So hopefully I didn't bookmark a project someone else is trying to judge the quality of based on stars. Who knows how much technical debt or damage I've caused because GitHub doesn't have a bookmark feature that isn't gamified.

    > I keep bumping into the fact that using mentally cheap signals of quality (such as stars or DL counts) almost never indicates the quality of the thing itself.

Astro.js, VueUse, Quasar (it's ugly, but amazing).

It seems like a pretty niche project with am enthusiastic following. Like Elm, but even more niche.

“…relies on the data source being able to seek backwards on its changelog. But Postgres throws changelogs away once they're consumed, so the Postgres data source can't support this operation”

Dan’s understanding is incorrect, Postgres logical replication allows each consumer to maintain a bookmark in the WAL, and it will retain the WAL until you acknowledge receipt of a portion and advance the bookmark. Evidently, he tried our product briefly, had an issue or thought he had an issue, investigated the issue briefly and came to the conclusion that he understood the technology better than people who have spent years working on it.

Don’t get me wrong, it is absolutely possible for the experts to be wrong and one smart guy to be right. But at least part of what’s going on in this post is an arrogant guy who thinks he knows better than everyone, coming to snap conclusions that other people’s work is broken.

I don't know, but it sounds like you skipped over most of the reasons why the author was annoyed by Fivetran. You advertise "Connect data sources to PostgreSQL in minutes using Fivetran" but if Dan Luu -- who is certainly an intelligent and capable engineer -- and his coworkers can't figure out how to use your product correctly, and if your customer support also can't figure out why the sync breaks, then maybe this isn't mere customer 'arrogance'.

Dan Luu claims, among other things, to experience hundreds of software bugs per week. If you believe the things he writes then he's not at all representative of a normal customer.

Today, for me: * Firefox mobile didn't load the keyboard on a textbox making me kill and restart the app to get it to work. * Firefox mobile pull to refresh triggered while scrolling up * I ask Alexa to turn on some lights and she dinged like she did it but nothing happened. * I turned on my office light that has a routine to turn on a space heater and another light. Only the first light turned on * The Roomba got lost and ignored its keep our zone and ran into the Christmas tree skirt. * When I ran out to get groceries android auto wouldn't connect until I restarted the car. * On another errand, apple car play refused to play music even though it said it was. * A website told me I had unsaved changes and wanted to know if I wanted to navigate away from the page without saving.... While clicking the save button. * I got a letter in the mail from Amex telling me that they couldn't reach me by email and I needed to log into my account and pay a zero dollar balance. This is after I closed all my accounts months ago, I get two letters each month to sign into an account that was deleted to pay a bill that doesn't exist. * Octopi said it's webserver wasn't running, a refresh fixed it. * Build tools at work linked the wrong binary for some tools and I had to manually correct the symlinks. * Insert 10 or more bugs with pipe wire and pulse audio on Ubuntu.

I'm sure there's more, every day is like this. Yesterday I had a plethora of bugs trying to get screenshare and webcam streaming to work for a video conference despite working for a dry run a few minutes prior.

And right now, line breaks aren't working in this reply

Often times on HN it can be hard to distinguish between scoundrels/trolls/losers/etc… with ulterior motives and genuine people with accurate information, but for this case since Dan Luu has done a lot of credible actions, he should be given the benefit fo the doubt.

Unexpected behaviour is how I choose to define "bug".

It doesn't matter if the programmer intended it. It's still a bug if it behaves contrary to the user expectation.

It might be that the best resolution is better documentation / training, but it's still worth of a bug being raised to fix.

If a product doesn't meet a user's expectations it may be a poor fit, improper usage, or even a braindead terrible design, but these are not bugs.

Many of them could be software bugs, sure, but without actually figuring out what's going on and what the root cause is it's hard to tell.

I would say failing to deal with hardware error that strongly is a bug. Keyboard I'm pretty sure is a bug, I've had plenty of situations where the keyboard code locks up and needs app restarts. Auto not being auto would be a weird thing to lie about, otherwise it's a bug. "Specified by their processes" a process is an algorithm, sending incorrect messages for $0 could be an algorithm bug or an implementation bug but either way it's a bug in the software, it's not doing that because someone decided it actually should do that. It said the webserver wasn't running but it was, that's a bug if they didn't have a exact unlucky timing.

- Four involving a Vim upgrade being pushed by one tool and blocked by another; a broken shortcut causing gVim to load with an error message and no working menus - not fixed by uninstall/reinstall; unable to delete one of the Vim DLLs; Windows Explorer window lockup after right click (possibly it had the DLL loaded for Vim context menu icon?).

- FireFox regularly stops loading pages until I visit Help -> About FireFox and see that it's silently done an upgrade and is waiting to restart.

- Cloud service with SSO login working for months, stopped working with mysterious 'error processing your credentials' type message.

- and it has a broken 'send password reset email' feature, no email gets to me.

- checking for blocked emails in SaaS email filter, when I put my email in the 'Recipient' box of the search, Edge browser autofill puts my email address in the 'Sender' search field as well, incorrectly / uselessly.

- Edge browser autofill dropdowns routinely cover SaaS product's HTML/JS dropdown menus.

- New SaaS product which has a username/password login instead of SSO, but login goes to a prompt with only 'use SSO' (which we don't) or 'forgot my password' (after logging in successfully). Clicking 'use SSO' gets past that screen, I think it's internal between-different sub-sites SSO abstraction leaking to the customer.

- SaaS product offers 2-factor auth using Google Authenticator, if using the signup key instead of QR code it appears in a difficult-to-select text label. Entering an expired TOTP code shows an incorrect error saying the code can only be used once (Time Based OTP codes are valid for the time duration, often ~30 seconds for any number of uses).

- Saving details for that site in a SaaS password vault from a different vendor, an error like 'no saved password' popped up despite having a password saved.

- At least four different debugger errors - set breakpoint either not setting, or not breaking, or debugger crashing, or code crashing without error message until upgrading the runtime.

- Windows 'restart and update' over and over until 'check for updates' and it decided there weren't any, then the option disappeared.

- Dell laptop firmware upgrade which rebooted to upgrade firmwares, then said it failed.

- Always on VPN for work which stops passing traffic until the service is restarted, a couple of times a month.

- Backup software which runs for a week then fails, until rebooted. Vendor support blames the network.

- Monitoring software which reported the wrong datastore sizes from a VMware host until the host management services were restarted, one or both of them are bugged.

- Theater website's date range selection on mobile, I dropped down the start, slowly worked through year/month and tapped a day, dropped down end, again went through year/month and tapped a day after the start, 'apply' button was greyed out, tapped outside the calendar controls, and they hadn't registered the days. Retried and it worked second time.

- Multiple websites where the infinite scroll or 'load more' links stop working, including big ones like YouTube.

- SaaS product search finding an item for one coworker, not finding it for another coworker.

- SaaS ticketing system not finding the ticket ID typed into the search until the search is retried a couple of times, multiple times per day.

- SaaS ticketing system not handling session timeout properly and presenting a normal interface where everything looks fine until attempting to interact with it and then presenting an 'oops something went wrong' style error message instead of going to the login prompt.

- Switching back to VMware web client in a browser tab, finding it showing the "your session will expire in 20 seconds" countdown box, knowing that the session expired an hour or three ago, and the 'extend session' button doesn't work.

- SaaS password vault, VMware web client, not handling session timeout properly and presenting a normal interface where everything looks fine until attempting to interact with it and only then whisking away to the logon prompt.

- Secure Jumpbox tool doesn't save its settings between sessions, even though it explicitly has a 'make this the the default value' type option.

- Secure jumpbox tool presents multiple screen resolution options which don't work, only some of them work.

- Database engine sync reporting different sync status in different views.

- Reddit discussion from 1 year ago with URL to the CDC website which is now a 404.

- iOS keyboard unreliable at opening the select-all/copy/paste popup dialog, often needs prompting and reprompting.

- iOS keyboard unreliable at starting swiping.

- After powering off phone and on again, torch GUI would switch 'on' but torch LED was not lit up, persisted for maybe 10 seconds after unlocking, as if there were internals still booting and the GUI was disconnected from control of the hardware.

- pfSense (OpenBSD-based) firewall upgrade process from old versions is broken.

- Computer game where the character turned invisible and then never turned visible again; glitchy bounds checks; glitch-teleported back to a starting point in the middle of a fight; unreliable animation on characters being enchanted; glitchy behaviour happening for an attack which doesn't have that behaviour. 'Disable background service' setting doesn't disable the background service. Network glitched and dropped back to the main menu not counting any of the progress made. Showed loading screen then didn't load. Network glitch and the fire button became the wrong way round - gun stopped firing when clicked, fired continuously when not clicked. Audio stopped working after RDP connection until game restarted.

- Proprietary software content to fill the disk with logfiles that aren't tidied up or limited by default.

- Search engines which don't search for what I type in and show something else instead. Try Google for "spirtling", top result is a Wiktionary page for it as an archaic spelling of using a Scottish porridge stirrer, other relevant results. DuckDuckGo result has changed it to "sporting" and are horse racing tips, then spirling and spiraling, then spirtling.

- New York Times' Connections game, on small screen phone the "one away" message appears in the wrong Z-order behind the top bar, unreadable unless tapped on to bring it to the front. After a game the 'register an account to track progress' popup partially blocks the 'close summary screen and look at the final game state' X button.

- Runaway memory use in Amazon.co.uk and YouTube.com tabs left open too long until FireFox restarted that 'about:processes' can't help with.

- Teams continuing ringing on cellphone after I've answered the call on laptop.

- Google StreetView glitches where it sometimes gets stuck at junctions and can't move forwards but can double-click-jump there instead.

- and if we count glitchy / unexpected / poor user interface or user experience as well.. Python's IDLE laggy keyboard input until latest version installed. Vendor renewal email which doesn't say what is being renewed in a useful way. Etsy online shop which encouraged me to click on 'related' links then gave me a captcha because they couldn't tell if I was human or robot. Inconsistent YouTube keyboard shortcuts. AWS's chat LLM gave incorrect instructions for how to achieve firewall rule change on AWS server. VoIP phone app which switches back to the signing-in animation every few minutes.

- The https://www.skyscanner.net/ date selection is poor (it's not clear that clicking 'Depart' also sets the 'Return' date until you try and get rid of the calendar so you can set the return date and find you cannot. You can (X) to clear the return date and the depart date stays configured so you can set a new return date, but you can't X out of the depart date to set a different one without clearing the return date as well and having to reset that. Once both dates are set, you can't adjust them; clicking the 'Return' box where the UX looks like you're setting a return date, when you click on a day it will clear both dates and then set the date you click on as the new departure date, getting in the way. The depart and return boxes have the same visual loko as the From/To boxes which you can type airport codes in, but this is misleading as you cannot type a date into the date boxes.

- https://calculator-online.net/sohcahtoa-calculator/ says "This calculator uses the SOH.CAH.TOA mnemonic method to solve the sides and angles of a right triangle. It provides step-by-step calculations using the SOHCAHTOA formula" but put in leg a=1 and sin ⍺=30 and it calculates that the hypotenuse is length 2, showing the step-by-step solution using Pythagoras' Theorem using the length of sides b and c that we haven't given and don't know.

And this isn't counting design I find confusing, documentation which is unclear, layout bugs where things don't visually align or the spacing is misleading or the organization is unhelpful, bugs in tabstop ordering, or not-really-bugs like Notepad++ lagged with a single line of 10MB of data, or EMACS lagged with a long line, etc.

- 'space' will play/pause the video if the main video has focus, or if most other controls are selected (e.g. volume), but if the settings are open then 'space' will change the setting and play/pause the video.

- holding 'space' when the main video has focus will 2x speed the video for the duration of the hold. Holding space when the 'play' button has focus will not do anything. Not double-speed, not repeatedly play/pause.

- 'k' will play/pause the video whatever control has the focus. Holding 'k' will stutter the video repeatedly switching play/pause. (Why?). Holding 'k' will never 2x speed the video. Space and K are both play/pause but implemented differently and inconsistently.

- Left/Right arrows rewind/fastforward by 5 seconds when most controls have focus, but when the volume slider has focus then left/right arrows change the volume. The Auto-play enabled/disabled control is also a left/right sliding one but the left/right arrows do not move that when it has focus.

- Up/Down arrows control the volume. Even when the volume slider has focus and left/right arrows are also controlling the volume. But wait, the up/down arrows don't control the volume when the settings are open; then they only move the selection in the settings. (Contrast with spacebar which activates settings and play/pauses at the same time, contrast with Return which sometimes does play/pause and does activate settings but does not do both at the same time).

- Page Up/Page Down with the video focused is browser page scroll. Click on the video position bar, the thin red one, and Page Up/Down do rewind/ffwd by 1 minute! With this focused, up/down arrows stop being volume control and now do the same rewind/ffwd as left/right arrows.

- j/l do rewind/ffwd like the left/right arrow keys, but they jump 10 seconds instead of 5 seconds.

- 'Return' activates a focused control so with play/pause focused it will do play/pause; it behaves almost like spacebar and it stutters the video like 'k' when held - but with the main video having the focus, Enter does not play/pause, it does nothing.

WHY are they so inconsistent?

I've seen some new libraries crop up for writing your own replication slot clients. I wouldn't use fivetran for PG.

Either you have a lot of data and fivetran will be too expensive or you don't, and you're better off just using a postgres OLAP plugin/extension.

Maybe it was because it was in beta, but I had a nightmare of a time with fivetrans API trying to coordinate connectors and destinations and git access.

When very smart people can't get your product to work as advertised, that's a problem with either the advertising, or the documentation, or maybe the default settings. Or maybe it needs the source data set up in a very specific way.

That kind of plays into the larger point of the essay that outsourcing this sort of thing still requires significant internal knowledge, and therefore may not be as cheap as it looks at first glance.

"part of what’s going on in this post is an arrogant guy who thinks he knows better than everyone, coming to snap conclusions that other people’s work is broken"

He's probably wrong about why it was broken, but it was broken.

And it's not exactly "arrogance" to give the best explanation you have, in a blog post about something else, while not mentioning the company name.

That's going too far. If the customer misunderstands the product or misreads the documentation, that's something worth addressing, but "broken" is not an informative way to describe it.

George however should not allowed to come to HN and start talking shit about random people who had the professional courtesy to never even mention the provider in question. The fact that his post isn't flagged, is highly upvoted, and dang hasn't swooped in to chastise him is a prime example of why HN is so fucking ridiculous and hypocritical.

Overall I think it's pretty unlikely that it wasn't broken.

> investigated the issue briefly

> coming to snap conclusions

Where exactly is the evidence that he tried your product only briefly and that he investigated briefly? I've read through it and don't see that anywhere.

After reading your comment, I lean towards you being the arrogant, thin-skinned one about your product and coming to snap conclusions about your customers who are paying for your product and having trouble with it and calling them arrogant instead of looking into why they are experiencing frustration with your product.

Perhaps Dan's conclusion was wrong, but the tone and wording of your response is just off putting and devoid of tact, empathy and teachability.

Something like "No, I don't believe it's broken because x, y and z. But I do see how the developer experience here is left wanting. Maybe we can improve it" would have been so much better.

It's not a five minute setup, but Dan doesn't write that the setup takes longer than five minutes - he writes that the design is fundamentally broken. Which it isn't, if you read the postgres manual. We're not even talking about the manual of the product he tried out for five minutes - we're talking about the manual of the database he's responsible for administering!

The overall point of the article is fine though. Original Commenter was nitpicking.

[1] https://www.postgresql.org/docs/current/warm-standby.html

Do you think it'd be reasonable for FiveTran to include this little tidbit in their setup documentation? I'm not talking about repeating the Postgres docs, but just a blurb about the need to do this kind of Postgres config?

That's an example of what I mean when I'm calling out georgewfraser to be humble and use Dan's feedback improve his product (in this case by improving the docs) instead of name calling his customers.

Ok, so Dan came to the wrong conclusion and was wrong to say the product was broken, but he had the professional courtesy to not name the company/product. George just attacks his character. Like another commenter mentioned, we don't even know for sure it was FiveTran. Yet, George just jumps in head first with guns blazing.

I was a longtime customer of fivetran who hit these sync issues constantly. Forced resyncs every other month. Was so thankful when our contract ended.

Dan's essay is dated 2022. It is now 2024, so maybe something has changed since then on the code path between Postgres and Fivetran to allow backtracking.

https://fivetran.com/docs/connectors/databases/postgresql/tr...

This doesn't match this:

> Syncing from Postrgres is the main offering (as in the offering with the most customers) from a leading data sync company, and we found that it would lose data, duplicate data, and corrupt data. After digging into it, it turns out that the product has a design that, among other issues, relies on the data source being able to seek backwards on its changelog. But Postgres throws changelogs away once they're consumed, so the Postgres data source can't support this operation. When their product attempts to do this and the operation fails, we end up with the sync getting "stuck", needing manual intervention from the vendor's operator and/or data loss. Since our data is still on Postgres, it's possible to recover from this by doing a full resync, but the data sync product tops out at 5MB/s for reasons that appear to be unknown to them, so a full resync can take days even on databases that aren't all that large. Resyncs will also silently drop and corrupt data, so multiple cycles of full resyncs followed by data integrity checks are sometimes necessary to recover from data corruption, which can take weeks. Despite being widely recommended and the leading product in the space, the product has a number of major design flaws that mean that it literally cannot work.

That description doesn't sound like _he_ briefly used your product, but that company he was working for used your product, found bugs and despite contacting support couldn't make it work. This doesn't read at all as a minor experiment that he didn't put in the time.

I don't understand why replication would need a backwards seek - are you saying it doesn't and he is mistaken on that?

It’s taken me far too long to internalize that the chances of someone making an (egregious) mistake in something I rely on to be correct are very much nonzero.

I see you’ve met my boss.

This post is a commentary on product quality issues, the underlying cost models (both goods and services), and the interplay with American culture. There's like 20+ company/product anecdotes in there - a mistake about one detail about one technical detail of one product is wildly uninteresting.

This just seems totally false on its face. If you've worked at the big guys you know they aren't magically smarter, they do very inefficient things frequently.

It's so intuitively false that I'd have to wonder about someone who thinks it's true.

And I don't mean that in a particularly bad way; most religions package a bunch of useful concepts (e.g., the golden rule) with some stuff that isn't literally true but does serve social and emotional needs in ways that the useful stuff gets passed down through the generations. As scholar Huston Smith put it, religion gives spirituality historical traction.

The notion that markets can drive efficiency is a valuable insight. But people for whom Econ 101 acts as a religion have a really hard time noticing when that effect gets swamped. This is pretty easy to spot these days because you'll find nominally pro-market people cheering oligopolies and monopolies, or getting upset at regulations that make markets more efficient. One easy test is how they feel about sustained high profits. To people who value markets for their ability to drive improvements through competition, that's a sign of something wrong, like insufficient price competition.

US businesses, though, do not. The desired state is monopoly, or, failing that, oligopoly with three or less major players. The US is there in cellular communications, web search, banking, drugstores, social networks, movies...

"Competition is for losers" - Thiel.

This quote is great but is not a call for building inefficient businesses. An inefficient business invites competition (which is for losers). An equally nice saying is "your margin is my opportunity".

Unless you reach the point where you have a monopoly and it's impossible to compete. At this point, you can become inefficient and still keep your monopoly. For a very long time, at least. And this is something we observe in software.

I'm not saying that "competition is for losers" is wrong. It's an excellent objective, to carve yourself a (large, comfortable) niche where you are the no-brainer solution. But it's no license to fall asleep.

Not because anything fundamental has changed about economics, but because baselines have shifted to the point that what we expect an efficient market to look like may be very different from what what people expected it to look like in the early 20th century. So, basically, people's definition of "efficient" has subtly changed.

A century or so ago market economies hadn't caught on to the same extent. In the 1950s you had Khrushchev coming to visit Iowa to learn about US agricultural productivity. He visited family farms, talked to people about how they did things, and then went back home to tell the USSR's farmers that they needed to plant corn everywhere, including in places where the climate is not even remotely suitable for growing corn. All this time talking to family farmers about how they make their own decisions about the best use of their own land, and he somehow still failed to pick up on the idea that maybe the secret ingredient in the sauce was that the USA generally let farmers run their own farms.

Sure, the USA's capitalist economy still had charlatans, including agricultural charlatans, and wasteful fads for bad ideas, rent-seeking behavior, pork barrel politics, etc. But maybe it was still easy to see that situation as efficient at the time, because one's reference for comparison included being able to see the greater amount of damage that a planned economy allowed a charlatan like Lysenko to do from his position of power within the Soviet Academy of Sciences.

Today most just pile into the megacaps and generally assume 'these guys are the biggest... they must be the best.' Sure, there's a small window of competition in the VC world where money piles into non-public companies for a few years before a winner is selected (often having nothing to do with having the best product/service or even being the most efficient or profitable... it's all about who scaled to the finish line the fastest) and either becomes the 800lb gorilla or gets gobbled up by one.

Probably.

A century ago, the companies which approached monopoly were in industries that had either huge economies of scale or very strong network effects. The United States Steel Corporation was an example of the first, and the Pennsylvania Railroad an example of the second. Even railroads were somewhat local monopolies - the Pennsylvania never merged with the Union Pacific. Not that the government at the time would have let them.

What changed?

- Better transport and communication. Selling to country-sized and world-sized markets became feasible.

- Computerization. Big companies used to have huge clerical plants of people pushing paper. Some of that paper pushing scaled faster than linearly, so the bigger the company, the worse it got. The administration system of big companies couldn't get out of its own way. It's forgotten now, but pre-computer, railroad companies had no idea where most of their shipments currently were. Paperwork was nailed to the side of boxcars and traveled with the load. As late as the 1960s, auto companies were struggling to figure out where their vehicles in transit on rail cars had gotten lost. Today the mechanics of tracking everything is a non-problem. Walmart, Target, and Amazon work fine. Planetary-scale companies are possible and common.

- Bigger banks. Until the 1980s, US banks could not operate across state lines. Until the 1990s, nationwide banks did not exist in the US. This limited large business access to borrowed capital. Most companies were primarily funded by equity, supported by national stock markets.