Documentation is the most valuable if it captures the design decision and intention at the time of the creation of the software, rather than the functionality of the software itself.

Its pointless to write "This function splits the input data into two equally sized chunks, multiplies each chunk with Y and then adds it together"

It makes more sense to write "The hardware X that this code runs on has a cache size of Y which makes this split necessary for optimal compute throughput".

This provides the next person an understanding of why the code looks as it does, and if it should be changed at this new moment in time, when perhaps new hardware is available.

I have seen so many design decisions being forgotten with time, and despite "documented code", a new engineer comes in and spends weeks trying to solve something that it inherently correct or is there for a good reason.

Excellent points on documenting the intention and design decisions, not just the "what".

I also use the following metaphor using an airplane:

There are three types of documentation:

1. 800 page manual with page 1 being "Congratulations on purchasing your 747!"

2. 10 page "How to change the oil in the engine"

3. 5 item checklist "How to deal with a fire in the engine"

Too many people think documentation is just one of the above. And if you are a developer, #1 feels overwhelming to write but the SREs may only want #3 (and may even help you write it).

This reminds me of the Diátaxis framework [0], which has a nice visual representation of the different types of documentation.

They have a fourth type they call "Explanation" that's "Here's why we made the flaps behave this way, and how that relates to the theory of aerodynamics"

[0] https://diataxis.fr/

Agreed. I remember hearing very good talk from someone working on Django documentations that was very close to what you describe. Cannot find the reference to the talk, but indeed this is how the documentation is organized: https://docs.djangoproject.com/en/5.2/#how-the-documentation...

Playbooks are one of the most important documentation artifacts, IMHO.

This is also why you should write your tests before you write the code. Far too many tests just test what the code already does. I don't care what it does, I care what it was supposed to do.

Wanted to share what I think is a great example of this https://github.com/facebook/rocksdb/blob/main/util/ribbon_al...

I think of this is answering the questions of what vs why. When what is being done is self-evident, that's readable code. When why it's being done is evident, that's good documentation. And, you need both.

This sort of documentation philosophy is part of US regulatory practices. The history of the enabling legislation is available online for all US regulations, as well as open comments made and agency responses to those comments.

“The most common reason for legislative history research is to determine the legislature's reason for creating or amending the law or the meaning of a term used in the law. This type of research finds the legislative intent.”

https://supportcenter.lexisnexis.com/app/answers/answer_view...

So context and motivation/decision and intention is preserved and made available to allow effective “debugging” of any US or State regulation.

It's the idea behind MADR (https://adr.github.io/madr/) and I wish more people used that.

"// Divides by 2" is the worst kind of comment and the unfortunate part is that thanks to "AI autocomplete" these will only be merged even more often [raises hand for forgetting to delete this instruction]

My company has a style guide that every public type/method/etc. must have a comment and it results in a ton of these comments for trivial methods.

Part of the problem is that I’ve needed to add comments like that to someone’s clever one-liners (probably mine).

Not a trick of the programming trade, but: life will not be clean, smooth and according to plan. Learn how to deal with things getting messy and derailed, and to accept that you "lost your streak" or whatever. Tomorrow is a new day, it is always ok to start over.

Do optimize for the long term, but also realize you could be dead by next morning.

One life trick I learned way too late is that you're allowed to have fun, on your trade or on the side. One should enjoy a few light-hearted or relaxed moments every week. For some reason I was convinced that one should seek productivity at every moment, which is the fastest way to burn out.

Another life trick I've learned is that you are also allowed to:

1. Say no and disagree with people

2. Walk away from any relationship (personal or professional)

3. Do what you like, believe what you like (as long as you are not harming yourself or others).

4. Move

5. Hire experts to work for you (if you can afford it)

6. Spend your money as you please

7. Engage in conversation with strangers

8. Drastically change your life (new career, new field of study, etc) 9. Talk to strangers.

It took me a long time to realize that being an adult means taking control of my life, make decisions and being okay with their consequences because _I_ made them.

For most people 8 is difficult because it took them ages to build up to a salary that pays as well as they do now, and they simply can't afford to do that again.

#9 isn't on its own line, and it's wrecking my need for consistency

Yet you are not bothered with the fact that it’s the same as #7

I completely agree.

Enjoying life here and now is such an important aspect. For me, bouldering is such an activity. It demands maximum focus on tiny details in order to climb a piece of rock, which is completely pointless to do. Pure fun.

I do bowling now and read your post as if you did as well :) Completely pointless and quite fun too.

If you're looking for a game with a point, try darts!

Or Kubb! It has a high silliness factor

I enjoy the solitude of bouldering by myself out in the forest. Solo Kubb is possible but wouldn't feel the same.

Some might object to this analogy, but I view life like investing in an index fund. It’s a diversified bet across many aspects of life and many projects (akin to how an index fund is a diversified bet across companies, sectors, geographies). Many of the bets will fail (much like many companies in the market will crash or delist), but some will deliver so much return they carry the rest, but you don’t know which ones and when ahead of time. It will be volatile, but if you persist and survive in the long run the expected outcome tends upwards.

Not trying to be rude towards you, but I am always a bit shocked and a bit sad hearing people take their life as an investment strategy. I understand your point of view, but life also needs to be lived, not experienced as a constant chart moving up and right. My brother was/is like that, who is calculating profits and opportunities when hanging out with his friends, and will stay friend with someone as long as he will get something out of that himself. I find it a bit weird.

However, I understand if you are coming from a poor background, most of your life resolves around how to improve it, so me not knowing anything about you, I am not judging you for this, just find it different.

That seems like a very different take on life than GP describes, that just happens to use the same analogy on the surface.

frontfor described "life as an investment account" and recommended a long-term strategy investing in index funds. zigman1 described "life as an investment account" and gave anecdotal evidence that a short-term "day-trading" strategy could be a bad idea.

They're both saying the same thing... although zigman1 didn't seem to realize it. (Perhaps through being unfamiliar with jargon like "index funds" and how that differs from day trading.)

"Variety is the spice of life" translated into neoliberalism.

> you could be dead by next morning

I feel like this is often overlooked in financial planning. Everything is aimed at planning for (early) retirement. Meanwhile, the '$1 today is more valuable than $1 tomorrow' trope isn't just true for monetary value, but also for the value $1 might bring.

$1 today can buy memories for the rest of your life. It can bring experiences with family members and friends. Having $1 million in the bank in 40 years will never buy back time you may have wasted working to save that money.

I'm not saying that people shouldn't save or invest money for the long term, but I do think more people should take into account what they lose today by only focusing on the long term... Especially when that long term might never come.

The “Die with Zero” book may go a bit overboard but I think the general message is a good one - do not OVER sacrifice today (and your youth) for a wealthy future you can’t enjoy.

The Bogleheads forum is full of “I scrimped and saved and now have millions but I can’t even bring myself to buy a used Toyota.”

It’s such an old problem it’s mentioned in Ecclesiastes 2:21-23!

Another way to look at it is you have time, effort, and savings; you need to balance them all because time will run out. A local college kid can do things that Buffett will never be able to do, because he has youth and years, and no billions can buy those.

Funny how when I read this, it reminded me of something I make an effort to remind myself of as much as I can; Hording is easily leads to waste - the feeling of having a lot and control or lack of, of drawing the line.

I pivoted from a career in security operations to go full time into tech and I find myself gathering too much information to solve a problem that actually doing the thing. In a wise and brotherly Nigerian tone Too much of anything no good

But Buffett can do things a poor old person cannot.

But can he go for a run?

My dad says basically "what you have seen and what you have eaten cannot be taken away from you".

He burnt a lot of money taking us around: car trips, ski trips, windsurfing, camping, cycling. He never regretted it and now I try to follow suit and spend good time with my kids before they grow up. I could have saved the money but the money physically won't be able to buy the same thing 10 years from now. It's a race against time in a sense.

In Mexico we say "Lo bailado, ¿quién te lo quita?" which roughly translates to "Who can take away what you've danced?"

Agree. In the end there is no simple answer. One should save for the future, think about the long run (in Shaggy's words). But one should also enjoy the present - but that damn tent so your family can go camping.

What should the balance be? Nobody knows. Making predictions is really hard, especially about the future.

Like my plumber said: the best time to change the old drains and pipes in your house is the day before they break.

I agree, we hear too much about how to save and not enough about how to spend properly.

Whenever I start thinking along these lines I end up imagining how damn difficult life must be for many others on planet Earth, and how easy we've got it.

Yes indeed. I am truly grateful having been born in a peaceful, rich country. No bombs dropping, no famine, no crazy dictator.

Sad reality is some peaceful countries are peaceful but invest so much to destabilize other nations and thus ruin it for many millions of people.

While we don't know what the purpose of life is, it is definitely not to beat death at the individual level.

Do things because you enjoy and care about them. This way if you have to start over it's yet another gift, because it's always your choice.

> Tomorrow is a new day

Today is a new day.

As immortal words of the master Oogway, "Yesterday is history, tomorrow is a mystery, but today is a gift. That is why it is called the present."

Or, as Yoyodyne Propulsion has it, “the future begins tomorrow!”

Just like all things, even when viewing the outcome of the long run, will see noise/errors along the way that just didnt have any real impact in the end.

An insanely nerdy analogy lost to most... A car may run rough at idle but strong at high RPM, because any ground loop noise is lost in the high frequency output.

Temporary solutions will become permanent solutions unless you explicitly hold yourself to fixing it within a certain time window because it becomes harder the longer time has passed.

This ranges from TODO's in codebases, to unpacked moving boxes, to replacing that sofa you got from your friend, to leaving a toxic relationship.

After a while you acclimatize to the situation and no longer see the issue, and unless something is really broken, it grows harder to motivate yourself to change.

This is maybe just me, but I need to remind myself of the opposite: All solutions are temporary and imperfect. So just do something, I can always go back and correct it.

For me this means that I'm allowed to store away 3 things from the moving box, even though I don't know where to put the rest yet. To invite others even though I don't know what to cook yet or to write a bad implementation quickly, instead of spending hours figuring out the best one.

I think a balance of perfectionism and can-do is important. But as people are predisposed differently, either advice might make sense in different circumstances.

> But as people are predisposed differently, either advice might make sense in different circumstances.

This is a very good point. Life is a balancing act. Different people may need the opposite advice. Or you may need the opposite advice at different times. Not just in this case, but in life in general.

I actually did not mean the post as a call to perfection/action but I can see why it could be seen as such. You can have a temporary solution that is good-enough for its purpose at a time. I advocate for the conscious effort to evaluate your intention so that you can feel a sense of agency around whatever happens afterwards because you are more likely to lose context, and the initial intention, as time passes.

Also, even after you learn this, other people will refuse to believe it, and will boldly agree to commit to months of work that is utterly pointless and even a net negative contribution to the world, because they will never do the other few months of work that was needed. And then when you say “no, we shouldn’t do this” to the 45th project plan, they get offended and swear on their own heart that they will force the second part to happen or, or, or nothing. And 6-12 months later they’ll do it again! Like fucking goldfish!

16 years ago when I was starting at my current employer as a contractor, they gave me some test data. I put it in a table called {domain}Temp. That table still exists, and about half of our production data flows through it. And yes, we've been talking about changing it for 15 years. (oof)

To be fair, everything is temporary.

This is a lesson that I've learned the hard way over the course of my life.

I do know a second way to prevent temporary solutions from becoming permanent solutions. You can just try out new things temporarily. Take your sofa example. If you choose to just try a different arrangement of furniture for a short time, that gives you can opportunity to remember to replace the sofa. The hard part is being willing to experiment with changes, but I find that it reframes things from fixing an old problem to trying something new, which can be more exciting and motivate additional improvements.

I'm kind of the opposite. I mark it as an issue or bullet somewhere to do and if it never happens or becomes a brain splinter, it probably never needed to happen.

This is decent. Make tickets for the todos ahead of time and make sure your PM gives them priority.

Oh damn.

I guess I've learned something.

Thanks :)

old codebases are always, plenty of these

All successful code bases have this ”problem”.

You become successful by making priorities. All parts of your product do not need to be perfect.

this is great advice!

The big one, for keeping my focus on the power of repeated, consistent action, and prioritizing my "future selves":

What is likely to happen if I do (or don't do) this thing one thousand days (or times) in a row?

Examples:

- exercising 2h per day and eating right --> I'm going to look and feel great and my health will be far better than that of my peers

- Should I buy these cookies along with the rest of my groceries? If I do that 1,000 grocery trips in a row …

- spending 30+ minutes per day reading the highest quality material I can find; taking notes; and figuring out ways to implement the knowledge and ideas I gain --> …

I have a similar thing but for putting purchases in context.

If I buy a good quality pair of jeans and I wear them 200 days every year for multiple years, its cost per wear is negligible even if I pay 100-200€ for them.

On the other hand a pair of fancy suit pants I wear for that one wedding and maybe Grandma's birthday is going to have a massive cost per wear -> no need to get the top shelf stuff, maybe just rent? Grandma is fine with business casual so I can go in with a jeans and blazer =)

The same works with subscriptions. Youtube Family isn't cheap - but we watch so much Youtube content that just the ability to skip all ads on every platform without addons or special clients is easily worth the time saved.

(It's also interesting that kids under 13 can't be enrolled to a Family on Youtube and thus are inundated with unskippable ads...)

> (It's also interesting that kids under 13 can't be enrolled to a Family on Youtube and thus are inundated with unskippable ads...)

Welp, looks like the addons and special clients are useful after all x)

I like your approach to relating your daily choices to your broader life goals.

I think you'd like Atomic Habits [1] if you haven't read it already.

[1] https://www.amazon.com/Atomic-Habits-Proven-Build-Break/dp/0...

This is a good one. What's gonna happen if I respond to comments on HN for 1000 days in a row? Uh oh.

You will have high reputation and be able to change the background colour, down vote comments etc.

Cute women who before didn't give you the time of the day will suddenly start paying you attention, slipping you their phone number, asking what you're up to on the weekend.

It takes altogether too long for many people to find out that they have a body.

I agree, it's not until something breaks that people realize they need to take care of their body.

how do you arrive at 2hrs a day? AFAICT 1hr a day gives you 90-95% benefit of 2hrs.

To put it another way, comparing to 2hrs

    15mins = 50% benefit
    30mins = 75% benefit
    60mins = 90-95% benefit
    120mins = 100%

Everyone has their own thresholds but paying double for 5-10% more doesn't seem worth it to me personally.

Real-life experience (over several decades). For me, one hour of even intense exercise is good but not enough to get me to, and keep me at, a point near my personal peak athletic performance. YMMV.

The secret is to have a very low peak. More of an athletic knoll really.

Is this based on personal experience, or do you have a source?

My own science-based impression was that it's more important to break up sedentary time with activity, and reach higher than mild physical exertion at least a bit, than it is to target any specific amount of time.

> I'm going to look and feel great and my health will be far better than that of my peers

Comparing your health (or anything to that matter) to your peers is a great way to boost and maintain insecurities. There is always gonna be a healthier, sexier, younger and so on. Good luck chasing that.

At risk of sounding very banal, comparing yourself to your version from yesterday is the best approach.

This is a fair point and I generally agree with you. That said, in this example I'm really using "peers" as a stand-in for where I or someone else in a similar situation will be with little or no regular exercise. (I'm a lawyer in Florida.)

I would expect less ambiguous definition from a lawyer :) (Please take it as a joke)

With the amendment it stands corrected indeed. Just added to make sure some people wouldnt take the idea of comparing themselves to peers as a generally good one.

Most problems (including analytically intractible ones) can be modeled with a relatively simple monte-carlo simulation. Most simple monte-carlo simulations can be fully implemented in a spreadsheet.

Using timing coincidences in particle physics experiments is incredibly powerful. If multiple products from the same reaction can be measured at once, it's usually worth looking into.

Circular saws using wood cutting blades with carbide teeth can cut aluminum plates.

You can handle and attach atomically thin metal foils to things by floating them on water.

Use library search tools and academic databases. They are entirely superior to web search and AI.

I feel like the Monte-Carlo simulation modeling trick is one I picked up intuitively but only recently heard formalized. Do you (or anyone else) have a list of example problems that are solved in this way? Like a compendium of case studies on how to apply this trick to real world problems?

_How to Measure Anything_ by Douglas Hubbard includes a chapter on Monte Carlo simulations and comes with downloadable Excel examples: https://www.howtomeasureanything.com/3rd-edition/ (scroll down to Ch. 6)

The main example is, you're considering leasing new equipment that might save you money. What's the risk that it will actually cost more, considering various ranges of potential numbers (and distributions)?

I think it's harder to apply to software since there are more unknowns (or the unknowns are fatter-tailed) but I still liked the book just for the philosophical framing at the beginning: you want to the measure things because they help you make decisions; you don't need perfect measurements since reducing the range of uncertainty is often enough to make the decision.

> I think it's harder to apply to software since there are more unknowns (or the unknowns are fatter-tailed)

Talking purely in agile software development, there is an idea of "Flow Metrics" [1] to use for estimation, basically boiling it down to "How many stories can we finish in the next timeframe (Sprint or whatever), and what is the uncertainty associated with this number?" I really like the idea, but haven't been able yet to test it.

[1]: https://www.prokanban.org/scrum-flow-metrics

A simple example that highlights the strength of this method: a 137Cs point source is at the origin. A detector consisting of a right cylinder at arbitrary distance and orientation is nearby. What is the solid angle?

There may exist an analytical solution for this, but I wouldn't trust myself to derive it correctly. It would certainly be a huge mess.

If we add that the source is also a right cylinder instead of point source, and we want to add first order attenuation of emitted gammas by the source itself, the spreadsheet becomes only a bit more complex, but there will not be a pen and paper equation solution.

In this example every row of the spreadsheet would represent a hypothetical ray. One could randomly choose a location in the source, a random trajectory, and check if the photon intersects the detector. An alternative approach would be randomly choosing points in both target and detector, then doing additional math.

The results are recovered by making histograms and computing stats on the outputs of all the rows. You probably need a few thousand for most things at least. Remember roughly speaking 10k hits gets you ~1% statistics.

What annoys me with Monte Carlo methods is trying to get self-consistent statistics at multiple parameter values. E.g., in your example, what is the derivative of the solid angle as the detector is moved? Or more generally, if we have some 'net goodness' measure for a system depending on parameters, how can we efficiently maximize it, when simulations are noisy and basins are shallow?

My understanding is that these sorts of questions come up in ML, and there are ways of dealing with it, but they can't converge nearly as fast as simple iterations like Newton's method. Even if I have to take a series approximation instead of a simple formula, I'll be able to use autodiff (or at worst, symbolic differentiation) to get quick and precise answers to these questions.

not sure if it counts as a list of case studies, but a relevant and recent video nonetheless https://www.youtube.com/watch?v=KZeIEiBrT_w

also in general bayesian statistics

random medium article: https://medium.com/pythoneers/monte-carlo-simulation-ideas-a...

Not a list, but a street fighting tactic is to apply this to finding counterexamples. Don't know whether a thing is true or false? Generate a bunch of random examples and check whether they all agree. Do this before you commit any energy to theoretical analysis. Obviously not a proof, but possibly gives a counterexample faster than you can think of one.

On the other hand, if you find yourself running 1hour+ numpy simulations for days on end you might want to consider an analytical approach.

Think of hard problems where sampling of outcomes could give you a better understanding

what do you consider top tier library search tools and academic databases these days?

I've used various commercial databases over the years. Some popular commercial databases relevant to HN readers include Web of Science, Scopus, and Engineering Village. When I worked at the USPTO, I used the less popular database Dialog, which I preferred. To my knowledge, none of these are available direct to consumers. I've only been able to get access from places with subscriptions. Some university libraries allow visitors where you can use these databases for free on-site.

I would call these databases complementary, not "entirely superior". There are two main advantages. One is that these databases will contain many things that you can't find on Google. The second advantage of these databases is that they are designed for advanced searchers and have more powerful query languages. Google on the other hand is dumbed down and will try to guess what you want, often doing a poor job. You can get very specific on these databases in ways that you can't with Google.

Related: I'm somewhat fascinated by more specialized bibliographic databases because they often contain things that can't be found on Google or the major commercial databases I listed above. I started keeping a list of them. https://github.com/btrettel/specialized-bibs

On a related note, if you studied and got a degree at a university, check if they have an alumni program. I pay a small yearly fee that lets me access the university's academic databases and their VPN, so I get some other perks as it looks like I'm connected through eduroam.

> none of these are available direct to consumers

This is the big problem with this "trick of the trade". It's not a trick, it's gatekeeping that excludes anyone without a university affiliation.

reminds me that my first taste of the internet was the days where you pretty much had t have a university account. I used a friend's, and was blown away by what I found on Usenet

Super helpful, thanks! Used to be up on the Databases when in school years ago, but it’s been so long. I wouldn’t actually know which ones are any good anymore did recognize scopus rest are new to me

even wood tipped circular saw can cut Al plate...

and academic/library search is indeed so underrated!

Concerning professional software development:

Customers buy solutions. They don't buy specifications documents, they don't buy elegant code, they don't buy automatic test coverage. All of these are useful only as long as they help you deliver solutions to customers faster. Any effort more is better spent on actually building solutions for customers. Maintainability is nearly top priority; real top priority is to have something to maintain in the first place.

Corollary: quick-and-dirty exist for a reason. It's easy to say that quick-and-dirty is just dirty, and it's true: in the long run, quick-and-dirty takes longer than the-right-thing. But, there is a but: time doesn't always have the same value! Time just before a deadline is much more precious than the time after the deadline, when you can relax a bit and fix things before the next stress wave. Have it working and ship it now; we'll get it in better shape next week, while the customer will be happy playing with their new toy.

And telling a customer that what they want a) won't work and b) is not cost-efficient to build can also be a good idea.

You might "lose" a customer, but word will travel that you won't build crap that wastes clients money. Which in turn might bring in more business later on.

Also when building a quick-and-dirty solution, make it LOOK dirty.

A fancy UI with a janky back-end will look like polished software to the customer, keep the user experience in sync with the code progress.

If your backend is WIP, the UI should be only unstyled plain HTML components when demoing it to the customer. If there is no UI, the backend should print all kinds of janky-lookig debug messages that make it clear it's not ready for production.

> Absolutely agree. It can be so difficult to explain to a customer why you've put together a prototype in 6 weeks but then struggle to explain why it'll take another 3 years to complete

Good luck convincing commission-based sales people of this.

Solution is extremely simple: commission-based sales people can only sell what already exists.

Corollary: R&D must have the right of veto on any promise to any customer.

Don't hire them. Commission-based sales people can only be used to sell transactional products.

> quick-and-dirty exist for a reason. It's easy to say that quick-and-dirty is just dirty, and it's true: in the long run, quick-and-dirty takes longer than the-right-thing

I've started to view this as "quick and dirty pays the bills for me to clean up later".

Yes, it'd be great to have an elegant solution right off the bat - but that's almost always going to take more time/effort. There's also a good chance what you build is completely wrong, so it might get thrown away.

Of course, there are exceptions to this. Particularly, for well-known, well defined things or safety critical things.

On that line, it's a good idea to optimize for the integral of business value delivered. It's better to deliver something partial and hack-y quickly so the business can start enjoying it rather than delivering all at once. Not to mention the invaluable interim feedback gathered from your users in the meantime.

Also, it's not enough to build something; you need to make sure people are using it, and using it right, or else you leave value on the table. Run trainings, monitor user engagement, talk to your users. Make sure what you built is used to its full potential and delivers the maximal impact.

> Talk [alternately: listen] to your users

Oh, how I wish the companies I work with did that. When I find one that does I'm immediately loyal for life... Or, well, until they IPO / exit - which, in my now repeated experience, is when they stop paying attention.

Definitely "documentation beyond "how to use this" is going to cost you more" is a policy I do for side business outside my day job. That said I do try to spend a bit of extra time to make the API easy to understand, well named, and expose only those things they need. But I just consider those good coding and time well spent when I have to revisit it. What lies beneath is not always be so pretty, depending on what the customer is paying.

> we'll get it in better shape next week, [...]

All of the above is only true if the quoted part is taken seriously and actually acted on.

Sure, I agree with you!

If you have a giant legacy code base with no unit tests, there are probably 2-3 unit tests you can write that will MASSIVELY improve delivery rate.

Example:

- owned a legacy system that deployed software to production

- there were "deployment windows" for different applications

- the "is now time to deploy app X?" logic was a giant rat's nest of if/then statements

- AND it used datetime.now() in the function

- that meant you had to wait for the window to actually hit to see if any changes worked

- changed the function to be can_deploy(current_time = datetime.now()) so you could pass in a time

- meant we could add unit tests

- changes to the logic went from taking two people 45 minutes hand tracing to <30 seconds

I tell this story whenever people say "I don't have time for unit tests!". I think they mean "I don't have time to write an entire suite of unit tests" which might be fair. But there is ALWAYS time to write a couple key tests.

You could also do this with freezegun or time-machine without changing the function signature. Although I guess your way is less off-putting for people who don't like tests.

good code may not look like you expect

- duplicate code isn't always bad

Sometimes* the same code copy/pasted or repeated is much easier to understand than pulling it all into a subroutine. It seems logical when writing code, but kills understanding when reading code. Predictable is better than compact a lot of times.

- code should fail early

Sometimes* "being robust", correcting errors or continuing after an error is the wrong thing to do. When something unexpected happens, failing, and failing quickly might be important.

- explicit is better than implicit

Sometimes* it is much better hardcode a list of things instead of constructing the list from whatever is there.

concrete example would be a makefile that compiles .c vs a makefile that compiles one.c two.c three.c. Hardcode the list. Make it hard fail if something is missing or something is added. Yeah, it is more work, but the madness it prevents is worthwhile.

there are probably thousands of these things. All are opinion, like camelcase sucks and indent should be 4, but not with tab characters.

* not always

Strongly agree with your second point, too many times I had to figure out issues caused by code trying to handle errors but actually hiding them and making it so much harder to debug. The typical antipattern: catching SomeErrorType just to log "error of that type occurred", thus hiding the associated error message and the stacktrace. Even worse: trying to catch every error types.

Don't catch errors unless doing something actually useful with them. Even then, it's often a good idea to re-raise.

I also believe people throw way too many exceptions. You should be able to be explicit about the outcome of an error case, and not just throw exceptions in a panic.

So alongside swallowed exceptions is throwing exceptions in scenarios that were entirely recoverable, and I think in both scenarios devs just don't understand their program enough and are choosing to panic, or ignore the problem.

I think people often throw exceptions rather than write the code that handles the situation more gracefully. If you knew it could happen, then is it really an exceptional circumstance?

I think it's impossible to come up with a good rule of thumb for error handling. Sometimes catch-and-log is what you want. Sometimes catch-and-reraise is what you want. Sometimes let-it-fail is what you want. I really cannot come up with a good argument that one of these is the better default. Languages should force us to consider which failure mode we want for every function that can possibly result in error.

The most egregious thing to me is the fact that most languages let any function throw any error it wants without requiring that be documented or part of the type system or anything.

100% agree. “Error” is as big and ambiguous as “success”. What should a successful function return? True / false? An object? A URI? Nothing?

There is no rule of thumb because the scenarios are too varied and context matters.

I try to make sure that the failure happens as close to the cause as possible. For instance, validate input when it's supplied rather than when it's used.

I’d like to add a corollary to this

- Sometimes what’s best for the system isn’t what’s best for your job

Fail fast is a good ethic but all to often I’ve encountered applications that are only to happy to limp along for the sake of optics. If that’s the prevailing sentiment where you work I’d recommend against violating it off your own bat.

> Sometimes* the same code copy/pasted or repeated is much easier to understand than pulling it all into a subroutine.

The Write Everything Twice principle is a much needed sanity check for absurd DRY practices. There are too many naive people arguing with a straight face that writing a base class and two specialization classes saves you the work of writing two classes.

That's one situation where it makes sense.

I had another recently where I need to implement an integration for a specific protocol. It's got a standard but most providers are not completely compliant and it's not mature enough to be exact. An expert consultant asked us to implement a pristine version and then subclass, add rules to change fields, behaviour etc. etc.

I suggested that it would make more sense to just copy/paste the implementation and make tweaks with comments that would accommodate the other providers. Worked like a charm and `diff`ing the files more or less gave us what's different. Got us from 0 to 60% without any problems.

I have started to gravitate toward ”write everything 100 times” or at least ten.

I’m thoroughly convinced DRY is the most ill-applied of all programming dogmas. At every place of work I have been I have run into bugs introduced by changes in one of the cases someone decided to ”dry up” several times. I have never seen bugs being caused by code not being DRY, and I cannot believe someone theoretically in the future maybe forgetting to include a bug fix in some case is worth the hassle.

In very few cases you are solving an actually generic problem. These are usually low hanging fruit already solved in a standard library or are available as an open source package in your eco system of choice. Your business logic is not general.

Minor nitpick that the point is not to save on the upfront effort, but to reduce maintenance burden. If you need to make a change that touches multiple endpoints, then it's usually easier and safer if you only have to make that change in a single place.

> Minor nitpick that the point is not to save on the upfront effort, but to reduce maintenance burden.

There are multiple benefits beyond that. Perhaps the most important one is preventing premature abstractions, which means not only avoiding all the work required to roll them out but also avoiding the work maintaining all the technical debt it creates.

Sure, but not every abstraction is premature; with experience and care, it's possible to walk the line between too much and too little, and premature abstractions can be easier to undo than overdue abstractions (you can always do the copy-pasting after the fact, but if you have two classes that should have common code but have evolved in slightly different directions due to lack of oversight, it can be very difficult to unpick that cleanly)

> Sure, but not every abstraction is premature;

How can you tell if what you're task to do is to duplicate a class?

The wisdom in WET is that it prevents the accidental complexity created by premature abstractions while not preventing the introduction of abstractions later. In the process, development work is also simpler and lower in risk profile because duplicating code only causes the introduction of an independent code path while preventing regressions. Naive interpretations of DRY are renowned to needlessly cause problems due to those mistakes.

> The wisdom in WET is that it prevents the accidental complexity created by premature abstractions

Yes, but...

> while not preventing the introduction of abstractions later.

...not necessarily.

A facetious example to illustrate the point:

I am required to implement DogWidgets and CatWidgets and I have two choices - separate the Widgetness into some kind of base class, or copy-paste.

I take the latter approach. The code is initially easier to read and maintain due, as you say, to reduced cognitive load. Fast forward two years; I'm no longer on the project. Now Joe Junior Developer comes along and over a period of several months, is asked to make a series of changes only to CatWidgets; some of these pertain to Widgetness, but Joe is not experienced or familiar enough with the code to understand that, and anyway has not been told to change DogWidget, so we eventually end up with two divergent implementations of Widgetness. Then somewhat later, Jolene Developer joins the project and is asked to fix an intermittent bug in CatWidget. The bug turns out to be partly due to the original Widget implementation, and partly due to Joe's changes - now Jolene not only has to fix CatWidget, but also examine DogWidget carefully to determine if a) the bug affects DogWidget too, and how, and b) whether or not to fix it, and how (which may be different from the CatWidget fix).

Somewhat later still, the business expands into the BirdWidget market. Rolling CatWidget's and DogWidget's Widgetness back together into a base class is now a very difficult refactoring job that is determined to take twice as long as just copy-pasting DogWidget and making Bird changes; doing the right thing by the code doesn't make economic sense. Now we have three problems.

If I chose instead to implement Widget, and derive DogWidget and CatWidget, then yes, I am imposing some additional burden on future developers to learn how the code is structured. However, now every time a change is made to CatWidget, if the design is good then it should be fairly obvious whether it's a change to Widgetness, or a change to Catness. Bug fixes and improvements to Widget now automatically filter down to both (and all future) Widgets; if Widgets need a colour, then even if we only want to give it to CatWidgets at the moment, we can stub it out or provide a default value for DogWidget, and that thought process also forces us to examine our design instead of ignoring DogWidget altogether, which helps to keep the design current and relevant. Designs should evolve over time.

If one day we get sick of this code structure and wish we had taken the other route, we can quickly copy-paste and completely fork CatWidget with basically no risk of regression, and minimal effort. BirdWidget is obviously trivial to implement. New Widget safety legislation can be implemented once cleanly, instead of three times in three slightly different ways.

> Naive interpretations of DRY are renowned to needlessly cause problems due to those mistakes.

Naive anything is problematic. DRY and WET are tools to be deployed when it is appropriate, not rules to be followed dogmatically; as I said there's a line to walk. As to your initial question:

> How can you tell if what you're task to do is to duplicate a class?

As far as I know there is again no hard and fast rule - it's a heuristic that we develop over the course of our careers: "is this likely to change in the future, and in what ways?" With experience, one can learn to foresee that the business might branch out into the BirdWidget market, and with care, develop code that supports that possible future without being too tied up in abstraction right now. Of course one can go too far, although as I outlined above, I personally feel that 'a bit too far' is overall better than 'not quite far enough', at least in the long run. I've personally spent countless hours trying to untangle legacy WET code compared with much less time necessary to understand DRY code.

> (...) so we eventually end up with two divergent implementations of Widgetness.

That means they were never the same implementation to begin with.

> Then somewhat later, Jolene Developer joins the project and is asked to fix an intermittent bug in CatWidget. The bug turns out to be partly due to the original Widget implementation, and partly due to Joe's changes (...)

Irrelevant. It means you have two components that are buggy. Create two big tickets and tackle them whenever the team has capacity.

How many months or years have passed since your hypothetical initial refactoring?

That's the problem with mindless fundamentalisme involving DRY. Throwing complexity around and tightly coupling components that in practice have no relationship other than superficial similarities are mistakes that only create technical debt in the long run.

> As far as I know there is again no hard and fast rule - it's a heuristic that we develop over the course of our careers: "is this likely to change in the future, and in what ways?"

That's the mistake you're making: fooling yourself into believing that mindlessly throwing complexity around prevents issues and future proofs implementations. It doesn't. You are only increasing the burden to maintain a project without bringing in any tangible positive tradeoff. YAGNI is a battle-tested guideline, which you're violating. Even in your example the introduction of a new widget meant bugs affecting it were either only affecting it, thus lower in risk profile, or already in the original component, which means low risk as no one stopped them. Why are you choosing to make your life harder?

OK, well, all I will add is that I've been where you are on this now, and I've also been an architecture astronaut, and now I'm somewhere in the middle. This is where I've landed based on long experience. I know which code I prefer to maintain, and it's not the big ball of mud that, in my experience, inevitably results from insufficiently abstracted designs. I'm absolutely not making my life harder; I'm making life much easier for my future self, and this is borne out by actual experience on projects I've worked on.

However as I said, there's no hard and fast rules, so, you do you.

The Rule of Three: “When you’ve written the same code for the third time, that’s when you should stop and refactor it into an abstraction.”

I think it's from Martin Fowler's book Refactoring. (It's been a while).

The gist of it is that with one or two cases you might create the wrong abstraction. When you have three, you can abstract away the right bits without creating a monstrosity of optional parameters and booleans.

> The Rule of Three: “When you’ve written the same code for the third time, that’s when you should stop and refactor it into an abstraction.”

Small but very important change: "[...] that's when you should stop and consider refactor it into an abstraction". Sometimes it's needed, sometimes it's not, dogmatic following of that rule lacks the required nuance for real-life programming.

“If it ain’t broke don’t fix it”

I don’t believe it’s wise to add another iteration to working code just for the sake of aesthetics.

There will come a point where you will have to write a fourth implementation and at that point you really should be considering abstraction and reuse of your three earlier known good implementations.

There’s also an argument for abstraction instead of having three different people write three different implementations simultaneously. It lets you stub it out to unblock multiple teams, and centralize learning/iteration. Still imperfect and not always the solution, but worth considering in a move-fast environment.

I like to think of it as that; you should have principles, such as trying to write DRY code, but any principle taken too far will end up being a bad thing. Don't be an extremist. Practice a sensible balance in everything you do.

This goes beyond programming as well, I think it goes for most things in life.

In every trade or art you start as an apprentice. That is the time when you learn the basics, the rules, the best-practices. When you have mastered the state of the art, you are a master. You know when to apply which rule and tactic to create masterful artifacts. The next step is to learn when you should break the rules and general wisdom. That is where true wisdom starts.

100%, on all of these. In basically every job where I worked with more than 5 people, I have regretted too many default arguments in functions that were trying to be overly dry, and code that tried too hard to not crash and carry on.

Regarding your first point, I feel it's important to understand Liskov's substitution principle, which is basically, the code might look the same now, but it could diverge in the future if the contextual semantics around that code are different. Effectively that heuristic has made me a little more hesitant to do a bunch of consolidation that could end up increasing complexity by a lot if you need a bunch of special cases.

Also sometimes copy paste is faster than trying to abstract a code block into something generalised

I think I had discussed every one of these items with my junior developer this week alone. Are you spying on me? Are we the same person?

Git bisect. For certain classes of software regressions, it makes life easier. Many people (me) don’t know this feature exists and end up reinventing the wheel on their own. https://git-scm.com/docs/git-bisect

Whenever people try to argue that small and meaningful commits do not matter, I introduce them to git bisect. I’ve yet to meet someone familiar with it that does not agree.

I have used git bisect twice in 15 years and seen it used maybe three other times.

I have burned hours upon hours on rebase conflicts over the years.

I have settled on squash merging feature branches into main and no force pushes on anything shared, ever.

> I have burned hours upon hours on rebase conflicts over the years.

Hate to say, "You're using it wrong", but if you haven't repeated conflicts rebasing, then your commits are probably just not small enough.

A merge does not magically eliminate any of the conflicts either, you just do them all at once rather than point by point. And the smaller the changeset, the less likely you will have any conflict at all.

Small changes, sorted lists (so parallel additions rarely conflict), allowing trailing commas (again to allow for parallel addition), splitting dependency updates from structural changes (to allow wholesale removing them if they were done separately), ... there's plenty of little changes to how you put together work that make this workflow easier.

Is this true? If the upstream changes some lines of code once, but your branch changes them 5 times in separate commits, iterating on the correct change to those lines, you will have to resolve conflicts 5 times. And 4 of those times you'll be expending effort to merge the correct code from upstream with code you know isn't correct/final on your branch.

Likewise if multiple iterations on those lines were made upstream while your feature branch exists.

Fundamentally expending any effort to find a coherent combination of two pieces of code where one of them you know isn't going to remain is a waste of time.

The only advantage would be if those incremental changes actually make it easier to arrive at the final merge of the last changes on each branch but I'm not sure your above tips really guarantee that. IME it rarely happens.

i can count maybe 3 times in the last 15 years ive used git bisect and it didnt even catch the bug in at least one of those cases.

It's one of those things that sounds super cool and advanced and i thought id be using all the time when i first learned about it but in practice 99/100 times to see when and where to bug was introduced it it's quicker and easier to just track it down manually in the code.

On the other side of experiences - i have used git bisect at least 3 times just last week to track down source of regression. I was not the person who wrote the code originally or introduced the regression. All i knew that it worked in last version but not now. Bisect, do an easy fix, add the missing test and move on to next bug. No need to think hard.

I suspect it depends highly on the size of the code, number of commits, number of committers, etc; I can imagine it can be very useful in e.g. Linux when you're trying to find when something was introduced when that may have been 15 years ago in a piece of code touched by hundreds of people / thousands of commits since then.

Even when the bisect points to a large commit, it's usually not too difficult to find the source of the regression. Sometimes I've had to go into the original PR and bisect on its commits which is a bit harder. It doesn't seem worth it to me to try to maintain commit quality within a PR when as long as I commit often enough it doesn't really make a difference.

Similar feelings about git worktree. Being able to check out multiple branches at once without having to deal with stash is a game changer.

And interactive rebase. Master interactive rebase and reflog until you can do it blindfolded.

Agreed with interactive rebase, but what of reflog? I've checked that for commit hashes when things have gone very wrong but I don't know many commands.

It allows you to observe and interact with the history of refs like HEAD, so superficially destructive operations like amending a commit can be recovered, reverted, etc. It's kind of like meta-git, allowing version control operations on the version control system. It's a lifesaver when you botch a merge or something and allows you to do "destructive" operations fearlessly (up to a point, you can munge a git repo pretty bad if you try hard enough).

I've only used reflog really to look up a hash and reset back to it. Is there anything else about it one should learn?

Interactive rebase is something I use a lot, but I should look into improving its ergonomics; sometimes I just want to apply the current changes to a previous commit. I did have a shortcut / script the one time but it was very finnicky and fragile.

https://github.com/mystor/git-revise is great for just this. Of course there's also `git commit --fixup=<commit>` and `git rebase --autosquash` if you don't want to install extra software. But the ergonomics of `git revise` are better.

I've also tried git-absorb before. It was buggy at least back then. Uninstalled when it crashed and lost my changes, and even reflog didn't help.

Very useful. What we always say after the fact is "we need more tests".

The amount of times I've used `git bisect` productively can be counted on one hand.

It's a useful tool, but sort of a tool of last resort.

Most people really cannot tell you what they want in any reasonable way. So expecting good specs for software without a very laborious interview and review process is pure wishful thinking. People "know what they like when they see it", so spend time rapid prototyping.

Smaller and more recent: iTerm has deep tmux support. Just do `tmux -CC` to start your session or `tmux -CC a` to attach to it and you don't have to memorise all the tmux commands.

Also half of those people will "tell" you what you need to build and you need to work backwards from there to figure out what their actual problem is before working forward again to figure out what you actually need to build.

Dealing with this at work right now. A newly promoted lead is struggling to delegate and all of the work coming out of his team right now is instructions on how to do what they believe they want. Very frustrating!

> Most people really cannot tell you what they want in any reasonable way.

But build them something and suddenly they know exactly what they want and what you built isnt it.

This is the origin story of agile software development.

You can’t avoid building what they don’t want, so might as well do so as fast and cheap as you can.

Then the flip side is 'oh i love what you built'

uhh wait a min that was an utter rubish version. 'no no no its great'. now you are kind of stuck with it.

There's a Germany saying "Auch nur mit Wasser kochen". Loosely translated, it means "they are also only cooking with water". What it means is that nobody knows shit and nobody is special.

That's a good saying.

I once heard: adults are children with older bodies. People seem/try to be rational but we are all so influenced or downright controlled by emotions. The desire to be perceived as rational is one such driving force. Many things we do since "this is the right thing to do" are in fact done to reduce (or control) our anxiety. In this sense we are still children.

Very relevant to most advice one finds on HN

how did you make your comment in gray?

Sick burn

Thanks guys, this was the best read of the morning.

Gray means the comment was downvoted

Only when you're viewing it from US, in other regions it looks grey

damn

In programming however, the skill and foundational understanding varies greatly, and while everyone is cooking with water, there are big differences how you cook. :P

Yeah, good saying but I think GP misinterpreted it. It does not mean “nobody knows shit”, it means “you have the same tools they do so any difference in outcome is due to skill that can be learned”.

Its a rejection of gear lust and similar (“I can’t possibly achieve that result without [toy that I want]”)

No, it's not what it means. It basically means "nobody is special". Whether that's true or not, is a different discussion, but that's how the saying is used.

I agree with the “nobody is special” take much more than the “nobody knows shit” take.

Don't tell this vibe coders without years of actual experience

In Quebec: we all fart through the same hole

Everyone's shit smells the same

Speak for yourself.

I work in embedded systems, and the best advice I can offer is: resist the urge to speculate when problems arise. Stay quiet, grab an oscilloscope, and start probing the problem area. Objective measurements beat conjecture every time. It's hard to argue with scope captures that clearly show what's happening. As Jack Ganssle says, "One test is worth a thousand opinions."

I thoroughly disagree with this sentiment.

In my experience, the most helpful approach to performing RCA on complicated systems involves several hours, if not days, of hypothesizing and modeling prior to test(s). The hypothesis guides the tests, and without a fully formed conjecture you’re practically guaranteed to fit your hypothesis to the data ex post facto. Not to mention that in complex systems there is usually 10 benign things wrong for every 1 real issue you might find - without a clear hypothesis, its easy to go chasing down rabbit holes with your testing.

That's a valid point. What I originally meant to convey is that when issues arise, people often assign blame and point fingers without any evidence, just based on their own feelings. It is important to gather objective evidence to support any claims. Sounds somewhat obvious but in my career I have found that people are very quick to blame issues on others when 15 minutes of testing would have gotten to the truth.

Very reasonable, I fully agree on that front

I think the GP is in a different world than you.

If you can grab an oscilloscope and gather meaningful data in 15 minutes, why would you spend several hours hypothesizing and modeling?

If you can't, then spending several hours or days modeling and hypothesizing is better than just guessing.

So I think that data beats informed opinions, but informed opinions beat pure guesses.

I agree with both of you. I think it’s sort of a hybrid and a spectrum of how much you do of each first.

When you test part of the circuit with the scope, you are using prior knowledge to determine which tool to use and where to test. You don’t just take measurements blindly. You could test a totally different part of the system because there might be some crazy coupling but you don’t. In this system it seems like taking the measurement is really cheap and a quick analysis about what to measure is likely to give relevant results.

In a different system it could be that measurements are expensive and it’s easy to measure something irrelevant. So there it’s worth doing more analysis before measurements.

I think both cases fight what I’ve heard called intellectual laziness. It’s sometimes hard to make yourself be intellectually honest and do the proper unbiased analysis and measuring for RCA. It’s also really easy to sit around and conjecture compared to taking the time to measure. It’s really easy for your brain to say “oh it’s always caused by this thing cuz it’s junk” and move on because you want to be done with it. Is this really the cause? Could there be nothing else causing it? Would you investigate this more if other people’s lives depended on this?

I learned about this model of viewing RCA from people who work on safety critical systems. It takes a lot of energy and time to be thorough and your brain will use shortcuts and confirmation bias. I ask myself if I’m being lazy because I want a certain answer. Can I be more thorough? Is there a measurement I know will be annoying so I’m avoiding it?

Another disagreeing voice, but I try to employ problem speculation when some problem arises. I'm working in a cross-company, cross-team project, where everyone's input interacts in interesting ways. When we come across a weird problem, getting folks together to ask, "what does x issue sound like its caused by?". This gets people thinking about where certain functionality lives, the boundary points between the functionality, and a way to test the hypothesis.

It's helped a dozen times so far essentially playing 20 questions and being able to point to the exact problem and have it resolved quickly.

This is a semi-embedded system. FGPAs, SoCs, drivers, userspace, userspace drivers, etc. Lots of stuff to go wrong, speculation gives a place to start.

Speculating is a great way to prioritizing what to investigate, but I've worked with many senior engineers (albeit not in embedded) that have made troubleshooting take longer because they disregarded potential causes based on pattern-matching against their past experiences.

This has become a personal debate for me recently, ever since I learned that there are several software luminaries who eschew debuggers (the equivalent of taking an oscilliscope probe to a piece of electronics).

I’ve always fallen on the side of debugging being about “isolate as narrowly as possible” and “don’t guess what’s happening when you can KNOW what’s happening”.

The arguments against this approach is that speculation and statically analyzing a system reinforces that system in your mind and makes you more effective overall in the long run, even if it may take longer to isolate a single defect.

I’ll stick with my debuggers, but I do agree that you can’t throw the baby out with the bathwater.

The modern extreme is asking Cursor’s AI agent “why is this broken?” I recently saw a relatively senior engineer joining a new company lean too heavily on Cursor to understand a company’s systems. They burned a lot of cycles getting poor answers. I think this is a far worse extreme.

For me, it's about being aware of the entire stack, and deliberate about which possibilities I am downplaying.

At a previous company, I was assigned a task to fix requests that were timing out for certain users. We knew those users had more data than the standard deviation, so the team lead created a ticket that was something like "Optimize SQL queries for...". Turns out the issue was our XML transformation pipeline (I don't miss this stack at all) was configured to spool to disk for any messages over a certain size.

Since I started by benchmarking the query, I realized fairly quickly that the slowness wasn't in the database; since I was familiar with all layers of our stack, I knew where else to look.

Instrumentation is vital as well. If you can get metrics and error information without having to gather and correlate it manually, it's much easier to gain context quickly.

To me, it's the method for deciding where I put the oscilloscope/debugger.

Without the speculation, where do you know where to put your breakpoint? If you have a crash, cool, start at the stack trace. If you don't crash but something is wrong, you have a far broader scope.

The speculation makes you think about what could logically cause the issue. Sometimes you can skip the actual debugging and logic your way to the exact line without much wasted time.

Its probably different depending on how much observability you have into the system.

Hardware, at least to me, seems impossible to debug from first principles, too many moving parts from phenomenon too tiny to see and from multiple different vendors.

Software is at a higher level of abstraction and you can associate a bug to some lines of code. Of course this means that you're writing way more code so the eventual complexity can grow to infinity by having like 4 different software systems have subtly different invariants that just causes a program to crash in a specific way.

Math proofs are the extreme end of this - technically all the words are there! Am i going to understand all of them? Maybe, but definitely not on the first pass.

Meh you can make the argument that if all the thoughts are in the abstract it becomes hard to debug again which is fair.

That doesn't mean any one is harder than the other and obviously between different problems in said disciplines you have different levels of observability. But yea idk

Implicit or explicit, you need a hypothesis to be able to start probing. Many issues can surely be found with an oscilloscope. Many others can't and an oscilloscope does not help in any way. It's experience that tells you which symptom indicates which class of issues this could be, so you use the right tool for debugging.

That's not to say that at some point you don't need to get your hands dirty. But it's equally important to balance that with thinking and theory building. It's whoever gets that balance right who will be most effective at debugging, not the one with the dirtiest hands.

Love this.

The most dangerous words during debug are: “…but it should work this way!” This is a mantra I try hard to instill in all EEs I mentor.

“Should” isn’t worth a damn to me. You test your way out of hardware bugs - you don’t logic your way out.

this works also in general purpose corporate programming.

Without speculation, what test do you decide to do?

Speculation is fine, but you need to ground it in reality.

> What Trick of the Trade took you too long to learn?

"Everything worth doing is worth doing badly"

And as a corollary, every complex system that works came from a simple system that works.

I learned this in programming, but now I apply it on everything from motorcycle maintenance, home appliance repair to parenting.

--

Often the easier way to fix a complex system is to pretend that it could be simpler and then reintroduce the complexity-inducing requirements.

I had a professor who taught debugging as a whole another skill from programming and used to say "Most of programming is starting from an empty editor and debugging until your code works".

The debugging "lab" in Java course (in the year 2000) was one of my transformational after-school classes - where I got a java program which fits within 2-3 pages of print code with a bug and was told to go find it in print for ~20 minutes, then given 40 minutes with a debugger instead.

> And as a corollary, every complex system that works came from a simple system that works.

"Do the simplest thing that works" is one of the few core architecture principles I stick my neck out for time and time again. Why write a simple function when you can spend a week accounting for every imagined corner case and implement modular expansion capabilities! Please... stop...

I empathize with the debugger story. If you're super deep in a language it makes sense to know the debugger inside and out. But stdout is universal and I've never been a specific language developer rather than being able to jump into whatever is needed.

Not always true. For example, if you work on your bike's brake, you should make sure the work is correct and well done. You dont want it stops working when cruising at 60mph.

I'll add, from Soul of a New Machine, "Not everything worth doing is worth doing well."

My new life motto, thank you very much! You just made a stranger's day better

It took me two decades to finally decide to memorize "... | awk '{print $1}'" as a command pipeline idiom for filtering stdout for the first column (... "awk '{print $2}'" for the second column, and so on).

All it required from me was to intentionally and manually type it down by hand instead of copy-pasting, on two purposeful occasions (within two minutes). Since then it's successfully saved in my brain.

I'd like to give a shoutout to jq for (non-JSON) text processing, and also as an almost-replacement for awk:

  echo "foo bar" | jq -Rr 'split(" ")[0]' # prints foo

I say 'almost', because jq right now can't invoke sub processes from within, unlike awk and bash. But otherwise, it's a fully fledged, functional, language.

Same! and jq's regexp functions are quite powerful for transforming text into something more structured:

  $ echo -e "10:20: hello\n20:39: world" | jq -cR 'capture("^(?<timestamp>.*): (?<message>.*)$")'
  {"timestamp":"10:20","message":"hello"}
  {"timestamp":"20:39","message":"world"}


  $ echo -e "10:20: hello\n20:39: world" | jq -Rn '[inputs | capture("(?<t>.*): (?<m>.*)$").m] | join(" ")'
  "hello world"

Also using inputs/0 etc in combination with reduce and foreach it's possible to process streams of text that might not end.

`jq -Rr 'split(" ")[0]'`? That would easily take me three decades to memorize ;-).

for selecting the second column, PID, from `ps aux`,

    ps aux | awk '{print $2}'

works for me, as gawk's default $FS field separator treats runs of whitespaces as a single separation of fields ( https://www.gnu.org/software/gawk/manual/html_node/Field-Spl... ), quite similar to "standard" Open Group/POSIX awk ( https://pubs.opengroup.org/onlinepubs/9799919799/utilities/a... ).

    ps aux | jq -Rr 'split(" ")[1]'

on the other hand doesn't work here due to `ps aux` adding a variable number of spaces for formatting.

    ps aux | jq -Rr 'split("\\s+"; null)[1]'

seems to work though.

Yes, in awk, conveniently, whitespace means a run of whitespace. It gets a bit verbose in jq for the same effect; but on the other hand, you get slicing - more slices than `cut` can give you.

These comments here about more or less clever text processing tools, each with their own syntax, feel like archaic hacks. Using something like Nushell or PowerShell makes this trivial.

E.g. for the `ps aux` example, using PowerShell, selecting a certain column:

  gps | select id

The output of gps is an array of objects. Selecting a property does this for all elements in the array.

People do this, but it feels a bit weird to start a whole new language interpreter which is much more powerful than the pitiful shell language we use, just to split a field. Why not write your whole program in awk, then? It's likely more efficient anyway.

The canonical shell tool to split fields is cut. Easy to use, simple to read.

For the very common use case to set variables to fields, use the shell built-in read. It uses the same IFS as the rest of the shell.

Why not `… | cut -f1 -d ” ”` though?

Because awk understands "columns" better.

echo '999 888' | awk '{print $2}'

prints 888

How is that different from

    echo '999 888' | cut -f2 -d " "

except for the default delimiter being space for awk?

   echo '999  888' | cut -f2 -d " " # notice two consecutive spaces

returns NULL.

There are invalid column separators for awk too. To handle consecutive spaces in cut you have squeeze them:

  echo '999  888' | tr -s " " | cut -f2 -d " "

Right, that would be expected though? I suppose awk is better for parsing formatted (padded) output.

How can it be "better" when it is exactly the same?

Speaking personally I find that I always use awk for printing (numbered) fields in shell-scripts, and when typing quick ad-hoc commands at the prompt.

The main reason for this is that it's easy to add a grep-like step there, without invoking another tool. So I can add a range-test, a literal-test, or something more complex than I could do with grep.

In short I can go from:

       awk '{ print $2 }'

To this, easily:

       awk '$1 == "en4:" {print $NF}'

Yet using cut, even with extra piping to tr or grep before, is probably over two times faster than awk, which may or may not matter depending on what you’re doing.

Now imagine what all the LLM copy pasting could do to our skills ..

I only learned this in the last five years: do less, automate less, do more by hand, and use the limited capability of the manual method to really choose projects that are worthwile, rather than aim for maximum efficiency.

Similar to this: if you want to optimize your productivity*, do so on a timescale of at least weeks if not months or years.

Simple example: Can you get more done working 12 hours a day than 8? Sure, for the first day. Second day maybe. But after weeks, you're worse off in one way or another.

It's easy to chase imaginary gains like automating repetitive tasks that don't actually materialize, but some basics like sleep, nutrition, happiness, etc are 100% going to affect you going forward.

* I actually hate that word, and prefer saying "effectiveness". Productivity implies the only objective is more, more, more, endlessly. Effectiveness opens up the possibility that you achieve better results with less.

In Finland we have flex hours or "working hours bank", basically you can do extra work now and get free time later. You can also go into negatives.

Usually the recommended range is -20 to +40

Some People(tm) go into the negatives and think it's easy to "just" do an extra few hours every day. It's not.

What I do is work ~15 minutes more every day so I bank about a hour a week, sometimes a bit more. It's a LOT easier and more manageable. Just sit on my computer 15 minutes earlier or if I'm at the office I take the later train back home.

This way I tend to have a day or two of flex hours banked if I need to take some quick time off.

Finland is a "high trust" country, isn't it? I can't see this concept working working when > 10% of the population would game the system to get more "off-the-record" free time.

The flex hours are usually only for office workers where performance can be measured - sometimes it replaces overtime.

So the employer can "suggest" you work longer hours now when there's a ton of stuff to do (like end of year rush in accounting or payroll) and then you take days off when it slows down.

But yea, it's not something that's easy to police.

it's not a 100 meters race, it's more like a marathon

This one took me years to develop: build minimal demos with all new technologies you want to use in a new project, _then_ begin thinking about how you’re going to build your project. You need to understand how a tech works and _if it even works the way you think it does_ before you use it. Otherwise, you’re gambling on hope and overconfidence.

Maybe I’m just dumb, but I always find that learning new tech while simultaneously trying to build with that new tech usually ends up in me rethinking the project repeatedly as I learn new tricks and techniques. I’ve dropped projects that I realized were too ambitious or just weren’t evolving right after months, years of effort. I’ve since learned that building needs to feel more like assembly than fabrication. You can dream, but it shouldn’t leave the whiteboard until _all_ of your technical assumptions not backed by experience are resolved into certainty. You move so much quicker and more predictably if you can predict success.

Understanding the concept of Opportunity Cost and how it applies to everything in life.

- Buying a house: Is this the best return I can get on a downpayment. (Spoiler: It is not).

- Accepting a specific job offer: Is this the best way to spend 8 hours a day?

- Not making a successful trade, is as much as a loss than losing money explicitely on a trade.

- If you own something, you should consider what could you do with it's cash value if you sold it instead.

- If you have a paid off home, could you sell it and get a better ROI with the cash equivalent and rent instead? (The answer is yes and you should do it).

I'd say you may be missing the concept of Risk-Adjusted Return, which changes a lot of those calculations.

And I would agree. There are a ton of second order effect and more complex concepts that I have learned about over the years. Opportunity cost is still the key one I go back to so often.

Home ownership is a huge economic subsidy for the upper-middle class in the US: mortgage interest deduction, cheap leveraged borrowing, real-estate investment, up to $500k of free capital gains, and an inflation hedge. It's pretty hard to beat compared to alternative investments.

That was the case initially.

Now the prices have skyrocketed due to offer and demand, and frankly because of the narrative that "Buying is always good", which ironically makes it a subpar investment.

Real estate is now a subpar investment. Especially today, and even with those advantages. It is also not diversified at all, absolutely not liquid and does not return even close to the SP500 (Even if you take the leverage into account, and leverages go both ways by the way...).

I've looked at this a bunch of times, from a pure return perspective, yeah equities usually win. But there's enough benefits to homeownership outside of that lens that I think either side can make a pretty compelling argument (I say this as someone who rents). It also highly depends on which market we're talking about.

The main thing is to look at it BOTH from a financial standpoint AND from a lifestyle choice, etc.

Too many people are too quick to dismiss renting because of the “conventional wisdom” - or think that only small apartments can be rented.

On the other hand, owning comes with comforts that can be hard to put dollar amounts on, for some.

yes - I often think about what I would do if I "won the lottery" or similarly got a significant amount of cash. Even though I have a low mortgage rate (refied in the middle of COVID), I'd probably still pay off my house. I know it makes absolutely no rational economic sense, but not having a mortgage payment and knowing that I own my house would give me a piece of mind that as you say is hard to put a dollar amount on.

I know there are people out there that like maximize returns with leverage and all of that - and the math all makes sense! - but I'd prefer the simplicity of not dealing with it.

Timing is huge part of it .. the real estate market goes in long term cycles, and if you buy at a good time you may do well over 10+ year timeframe, but if you buy at the peak then it may take that long just to break even if you sell (just in terms of house price, forgetting taxes/etc, and real-estate fees - 5-10% of price !).

Then of course you have all the expenses of home ownership, starting with real estate taxes ($20K/yr by me), maintenance (incl. costly items such as roof replacement), etc.

Deliberately investing in real estate, where you can choose the city/market, type of dwelling, need for renovation, etc, may have a higher chance of being profitable, although even there obviously buying at bottom vs top of the market makes a big difference. For people just buying a house to live in, conventional wisdom is not to expect it to be financially preferable (all told) than renting, even if sometimes - depending on timing - it may turn out to be.

A house is something that fulfills your direct needs, unlike S&P500. You 'miss the opportunity' in exhcange for feeling safe about your future, which leads to more rational choices long term.

Or less rational like tying yourself to a place, missing opportunities in professional and personal life or opportunities to experience a different life.

Buying early in life is in my view almost always wrong.

> If you have a paid off home, could you sell it and get a better ROI with the cash equivalent and rent instead? (The answer is yes and you should do it)

Could you explain how that makes sense? I can't see it working except if you are willing to accept a huge risk by putting the money into financial markets. Banks offer around 0% in my country, and with 4.5% inflation keeping cash just burns a hole in your pocket.

It’s a uniquely American (which can include Canada) situation - some very nice places to live have extremely strong rental inversion markets; where it costs $4k a month to rent a place that would easily have $10k a month in mortgage and property tax.

Landlords are willing to “subsidize” renters because appreciation has been so high. But it’s really only terribly high when leveraged and often the accounting is soft.

The buy/rent discussion isn’t a clear cut one, and it’s more than financial, but it’s certainly not a “buying always wins” scenario as many seem to think it is.

Yep, this is exactly the situation in the Seattle area. Here it makes little sense to take on a mortgage when renting is so comparably cheap. But go anywhere else like say Atlanta where rents are roughly equal to a mortgage payment and the opposite is true.

Generally you are right. VHCOL (Seattle, SF Bay Area), HCOL (California,...) and MCOL areas are better for renting and LCOL areas are better for buying.

There are two factors really:

- Rent to Buy ratio (or as you put it, rent compared to the mortgage)

- Speculation on house price growth.

If you sell the house you live in, you can keep $250k ($500k if you’re married) of capital gain without any federal taxes at all. So an equity investment would need to return 17.6% better just to match that tax advantage. This gets left out of a lot of comparisons in my experience.