Object Relational Mapping (ORM) tools, which focus on mapping between code based objects and SQL tables, often suffer from what is called the N+1 problem.
A naive ORM setup will often end up doing a 1 query to get a list of object it needs, and then perform N queries, one per object, usually fetching each object individually by ID or key.
So for example, if you wanted to see “all TVs by Samsung” on a consumer site, it would do 1 query to figure out the set of items that match, and then if say 200 items matched, it would do 200 queries to get those individual items.
ORMs are better at avoiding it these days, depending on the ORM or language, but it still can happen.
I dislike ORMs as much as the next ORM disliker, but people who are more comfortable in whatever the GP programming language is than SQL will write N+1 queries with or without an ORM.
Very true. But ORMs did make it particularly easy to trigger N+1 selects.
It used to be a very common pitfall - and often not at all obvious. You’d grab a collection of objects from the ORM, process them in a loop, and everything looked fine because the objects were already rehydrated in memory.
Then later, someone would access a property on a child object inside that loop. What looked like a simple property access would silently trigger a database query. The kicker was that this could be far removed from any obvious database access, so the person causing the issue often had no idea they were generating dozens (or hundreds) of extra queries.
This problem is associated with ORMs but the moment there's a get_user(id) function which does a select and you need to display a list of users someone will run it in a loop to generate the list and it will look like it's working until the user list gets long.
I really wish there was a way to compose SQL so you can actually write the dumb/obvious thing and it will run a single query. I talked with a dev once who seemed to have the beginnings of a system that could do this. It leveraged async and put composable queryish objects into a queue and kept track of what what callers needed what results, merged and executed the single query, and then returned the results. Obviously far from generalizable for arbitrary queries but it did seem
to work.
I think many ORMs can solve (some of) this these days.
e.g. for ActiveRecord there's ar_lazy_preloader[0] or goldiloader[1] which fix many N+1s by keeping track of a context: you load a set of User in one go, and when you do user.posts it will do a single query for all, and when you then access post.likes it will load all likes for those and so on. Or, if you get the records some other way, you add them to a shared context and then it works.
I defense of the application developer, it is very difficult to adopt set theory thinking which helps with SQL when you've never had any real education in this area, and it's tough to switch between it and the loop-oriented processing you're likely using in your application code for almost everyone. ORMs bridge this divide which is why they fall in the trap consistently. Often it's an acceptable trade-off for the value you get from the abstraction, but then you pay the price when you need to address the leak!
Also what you're looking for is called Microsoft Excel. Every one else is doing webshit UIs which can't be world-class by definition, because they hardly scale to real workloads :).
I've been involved in a lot of due diligence efforts, from the tech side but I've seen all angles of it as the deals are often fast and intense and the various teams have to often coordinate to a degree (tech, legal, financial, tax, etc).
It is fairly common for the people initiating the acquisition to really want to close it in a hurry, and they do due diligence only as a check mark in someone's list. As someone else here mentioned, there is enormous pressure to close, and any red flags are often redirected, reworded, or even occasionally just squashed.
The further away a company is from something like private equity, who does acquisitions like we eat breakfast every day, the more likely you are to see rushed and potentially botched due diligence. Someone like a big bank may well have the main proponent not know anything at all about acquisitions or due diligence, and just wants to "get 'er done".
It is also very common for people to come in after-the-fact and do a second diligence, and while doing that diligence to hear one or more people opening the conversation with "I warned them about this before the acquisition...".
At the end of the day, particularly in a big public corp, people are focused on their bonuses and total comp, and people like that aren't going into a due diligence looking for red flags and "no's".
I got started in the 1980s, and super-curious and technical people were the norm. We were incredibly strongly attracted to computers.
The first real growth in computers in that kind of era was Wall Street and banks. Wall Street in particular started paying huge bonuses to developers because it was clear that software could make huge piles of money. Then we started seeing more people joining in for the money who were not necessarily passionate about technology.
Then came the dot com era and bust, and then the rise of social media, FAANG, and absurd corporate valuations allowing ridiculous total comp to developers, and the needle moved even more towards money.
The net result is the curious and the passionate are still here, but our numbers are massively diluted.
I come places like here to find that passionate niche.
I live in the US in New Jersey, and here a big problem was the State flooded school districts with money during Covid with no material oversight of its spending.
The end result was huge increases in spending. But not on education. The money was spent on more MacBooks, more iPads, more buildings, more smart TVs, more consultants, more School Bullshit System as a Service, more scoreboards, more $50,000 signs in front of schools.
Meanwhile the good teachers are fleeing the system and test scores are plummeting as schools focus more on day care and “social justice”, and a declining emphasis on teaching core subjects and learning in general, coupled with social promotion where everybody gets a C or higher, and 80% of the school gets on the honor roll (spoiler alert: our district is not some outlier where 80% of the kids are geniuses).
Schools have very little to do with teaching, and really are just about baby sitting and trying to correct social issues.
Oh, and endless buckets of tax payer money with meaningless oversight.
> Schools have very little to do with teaching, and really are just about baby sitting and trying to correct social issues.
Is that wrong? The government takes away your kid for 12 years, every weekday all day, they might as well solve social issues in the country even if that means, say, kids are 1 year behind Asian kids, or their parents 30 years ago. If they figure out how to solve personal issues, that's even better.
I think there is a logical fallacy here. People assume that the only purpose of school is education. The more the education the better, even if that means deepening social issues, or making kids unhappy (BTW being a kid is like ~20% of someones life, not insignificant in itself). I think they assume it just because 'school' is called 'school', but I don't think the name of an object should determine its purpose.
- - -
When I look at the social issues in my country, I think the school system would be a very natural place to start to solve them (and arguably the current school system just worsens them). Even at the cost of "fall in reading and math scores".
I fully hear you on this. I miss the days where a simple phone call or email communication would occur when needed. Now it's a deluge of daily updates via 2 separate 'apps' for 2 different schools, and a requirement to login to 'app' or website to read the 'email' that they've sent out. Nevermind contacting someone that isn't directly associated with your child at the school -- Guess that's all need to know basis.
New Jersey is probably the most socioeconomically segregated state in the country, mostly based on its school districts. It has crazy real estate prices precisely so parents can get their children into specific, high-performing school districts. These districts bring the state average up very high, but best of luck if your district is in the bottom 50%.
> The money was spent on more MacBooks, more iPads, more buildings, more smart TVs, more consultants, more School Bullshit System as a Service, more scoreboards, more $50,000 signs in front of schools.
> “social justice”
So they spend all their effort on social justice, but spend none of their money on it? You should move to the South, you can pick a charter/magnate/whatever school with no special education and no busses. Keeps out the pesky blacks and retarded.
Somewhat of a pain though since my son has autism and services are pretty shit compared to where I lived in NJ
A term coined by consultants who don’t understand an industry who basically say “do the least possible thing that will work” because they don’t understand the domain and don’t understand what requirements are often non-negotiable table stakes of complexity you need to compete.
It reminded me of a Martin Fowler post where he was showing implementation of discounts in some system and advocating to just hard code the first discount in the method (literally getDiscount() {return 0.5}).
Even the most shallow analysis would show this was incredibly stupid and short sighted.
But this was the state of the art, or so we were told.
See also Ward Cunningham trying and failing to solve Sudoku using TDD.
The reality is most business domains are actually complex, and the one who best tackles that complexity up front can take home all the marbles.
A better explanation of this is on my blog todo list, but:
> Even the most shallow analysis would show this was incredibly stupid and short sighted.
Why? Yes, there's a high probability that this single line of code is ultimately wrong. But having it allows for testing the system (to ensure that getDiscount gets called, that the resulting price is between zero and the undiscounted price, etc.) and it can trivially be replaced when the actual discounting logic becomes known. Nothing can reasonably be called "short sighted" that doesn't actually limit you in the future.
> See also Ward Cunningham trying and failing to solve Sudoku using TDD.
It was Ron Jeffries who was struggling with Sudoku, not Ward Cunningham.
And he didn't struggle because of "doing simple things", but because he didn't actually know how to solve Sudoku, and rather than doing research (or trying a few by hand and reflecting on his own thought process) he expected to gain special insight into the solver by modeling the problem more accurately.
If he had actually known, then he would have just applied the same techniques to modeling the solver, and avoided all the embarrassment.
TDD is orthogonal to "doing the simplest thing", just preached by many of the same people. The problem isn't with writing tests for early iterations of the project that aren't anywhere near fully functional. The problem is that the making the tests pass doesn't actually teach you anything about what the next step of functionality is. You still have to actually think, and Jeffries didn't have a proper basis to ground his thinking.
Norvig's approach to the problem involved some clever hacks that aren't necessarily the best thing from a code maintainability standpoint. But he resisted the temptation to create a prematurely generalized system for solving constraint-propagation problems, trying to create some generic abstract representation of a "constraint" etc. In essence, that code didn't diverge from what the XP gurus preached; it just didn't follow their specific methods ("OO" modeling that's heavily based on creating new classes, TDD etc.).
I independently came to the same conclusions about how to solve the problem, however many years ago. Recently, I was reminded of the story (I think via https://news.ycombinator.com/item?id=42953168) and tried writing a solver from scratch as an exercise. My code was certainly not as "clever" as Norvig's, but only a bit longer and IMO much better factored.
"Hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature".
I used to avidly read all his stuff, and I remember 20ish years ago he decided to rename Inversion of Control to Dependency Injection. In doing so, and his accompany blog, he showed he didn't actually understand it at a deep level (and hence his poor renaming).
This feels similar. I know what he's trying to say, but he's just wrong. He's trying to say the LLM is hallucinating everything, but Fowler is missing is that Hallucination in LLM terms refers to a very specific negative behavior.
That's the point. It is meaningless. When it first coined, there were already detractors to the term, that it is an incorrect description of the phenomenon. But it stuck.
No, it’s just an attempt to pretend wrong outputs are some special case when really they aren’t. They aren’t imagine something that doesn’t exist they are just running the same process they do for everything else and it just didn’t work.
If you disagree then I would ask what exactly is the “specific behaviour” you’re talking about?
Somewhat related, I learned to be very cautious about LinkedIn and their job postings. I hit a couple that looked like regular companies and started to apply, but they were really fake job postings just go harvest your information. Even when you abort the process, it's too late. They take your info and put you on a zillion job sites automatically with endless email spam.
I still won't forgive them for the incident circa 2012 where they were sending out emails from your email account to everyone in your address book or recent contacts list in your own voice (!!) saying "hi $name, I joined linked in come network with me!".
And years after that, NextDoor did... and even used physical mail to spam people's real neighbors using the "inviting" user's name. Despicable, and even potentially dangerous in these sad times of polarized and unhinged trash.
Oh yeah... NextDoor also exposed people's exact physical addresses to all users by default for a year or two. I mean... that's just inexcusable and deliberate irresponsibility.
LinkedIn is the absolute worst. For one, there's a checkbox that is clicked by default to "Follow <company>", which essentially rewards a company for posting a job with thousands of followers (job applicants).
I imagine most companies just want followers on LinkedIn.
I don't get spam emails, but I get spam phone calls. Sometimes it happened suspiciously a few minutes before a call that's actually related to the position I applied for.
But a phone number is expected here where I live (Spain). Few companies seem to respond via email (some do, but it's rare). Everyone just wants you to give them your phone number so, if they decide to call you, they will do a surprise call a few weeks after you applied, at the most inconvenient moment, with no advance notice of any kind.
And since first impressions are important, not being available during that inconveniently-timed surprise screening call is probably a negative point at least on a subconscious level.
A naive ORM setup will often end up doing a 1 query to get a list of object it needs, and then perform N queries, one per object, usually fetching each object individually by ID or key.
So for example, if you wanted to see “all TVs by Samsung” on a consumer site, it would do 1 query to figure out the set of items that match, and then if say 200 items matched, it would do 200 queries to get those individual items.
ORMs are better at avoiding it these days, depending on the ORM or language, but it still can happen.
reply