Over the years I have realized that some comments are
needed and useful. These days I add comments when there
is something particularly tricky, either with the
implementation, or in the domain.
Ah, yes. Another convert. There are dozens of us. Dozens!!
Nothing makes me feel crazier than trying to get fellow engineers to comment their code. Code alone can only tell you the "what." If the "why" is not obvious, comment it.
Unit testing private methods.
I am always dismayed by engineers who vote a hard and unyielding "no" on this issue.
I guess ideally your private methods should not be tested directly. If you work at an ideal shop doing ideal things under ideal conditions, please let me know if you are hiring.
In all or most of these cases the tests for private code will hopefully be somewhat temporary; perhaps think of them as scaffolding used during construction or renovation.
- For example, perhaps you have a good reason to write the private methods first and you would like to make sure they are sound before proceeding.
- Perhaps you have a division of labor due to a time crunch and you are writing the private methods while somebody simultaneously writes the public methods.
- Perhaps you are encountering some thorny preexisting code with no test coverage and you would like to just make sure things work.
- Perhaps the public methods are undergoing a lot of flux and you would like to make sure the private methods do not suffer regressions during this flux
- Sometimes it's just easier to test private methods directly rather than indirectly via a public interface. Maybe this is a code smell, but also maybe you don't have time for a full refactor.
I'm still firmly in the "no" camp on testing private methods. By definition, anything your class does in the real world is done via its public interface; I'm not sure why I need to care about what's done under the covers.
I've actually moved further and further away from unit testing over the years (after being a pretty big TDD fan for a long time). In terms of bang for buck, integration tests across your public API are the best IMO. You're testing how your API is actually used. The problem historically has been that they're fragile to refactoring and difficult to run, but with the right tooling you can get around that.
I find the opposite wrt. refactoring - that is to say integration tests are _more_ robust to refactoring than unit tests, and that's one of the reasons I strongly prefer them.
Unit tests are deeply coupled to the internal structure of your system - refactoring often implies changing unit tests, which opens the door to bugs where the code and test both change to match each other.
As you say, integration tests validate your public API, which from a 'correctness' point of view is really the only thing you care about, not the internal structure of the system. That's why I love integration tests, you can make sweeping refractors without needing to change the tests, because the test will still tell you whether the behaviour of the whole system is correct.
Yes, especially for some unit testing is done like "atomic testing" (test the smallest possible part, if they could they would test individual lines), but turns out that the putting together is another crucial part with often not less complexity and where the corner cases come in. In that sense even not much like the unit vs integration testing difference & debates, because that line is blurry and just depends on your definition of an unit: It may be the single heavy algorithmic function, it may be the whole state machine, it may even be two heavily dependent and cooperating tasks in an embedded RTOS... and you always know you are in trouble if people discuss more what an unit test is and not is (instead of the value whatever thing provides).
I "love" btw those code bases rich with tests but 99% of the time not finding a bug, but breaking on the tiniest detail change.. not supporting refactoring like it should be but making it slow and everyone wanting not to refactor because of the crap test rat tail.
Integration tests are robust with respect to process flow/business logic, but (can be) fragile with respect to the mechanics of calling the API. Back In The Day that used to be a real issue because the tooling for writing API integration tests was separate from the tooling for writing the API itself.
These days (at least the way we do it) the integration test code gets refactored alongside the rest of the code, and compile-time errors catch 90% of issues.
That's part 1 of what sold me on integration tests. Part 2 is the ability to simply and efficiently spin up a DB per test, removing the need for mocking the persistence layer.
I’m wondering if the software world has something to learn from digital design here (for once), where there’s a huge emphasis on code coverage collection.
You really shouldn’t care how a function is tested. That there exists a test for that particular function isn’t particularly useful measure of how well the function is tested. So you need a methodology that tells you which of your functions have been covered by your tests. If it’s not covered you should make a judgement about how to get that coverage.
Getting your private methods covered by the public interface is good because it encourages you to write thorough tests for those public methods that covers all the ways the method might be used.
I agree a lot with this, and want to add that if you have a lot of complex private logic in a class it might make sense to extract an abstraction from it (if possible) and make it its own class for which you can then test the public (previously private) methods.
First thing I'd say is that there's no "one size fits all" approach. The right answer depends on the language, the project, the requirements, the amount of churn, the timeline, etc. So I 100% believe you when you say you've found success by moving the focus toward implementation tests. =)
But in my experiences, I've had major issues with a reliance on integration tests. This is almost exclusively in Rails.
- They're slow, because there wasn't an easy way to stub out the slow stuff
- If you stub out the slow stuff now you have no coverage for the stubbed stuff, unless you also have unit tests covering the guts of the app
- When integration tests fail, it can take a lot of work to investigate the cause, as opposed to granular unit tests
Our default approach is to mock and stub as little as possible, but of course there are things we just have to mock, like external service calls. However, we also have UI automation tests that run on a fully deployed and integrated system, which mostly covers that.
Slowness is only a problem when it's a problem, and can sometimes be solved with better tooling. I'd feel more confident in a system with 1000 integration tests that take 2 hours to run over one with 10,000 unit tests that take 10 minutes to run.
On your last point, this is positive and negative. Yes, it can be harder to find the issue, but IME integration tests actually find the issues that will cause things to break in production, and it's easier to find out why a test is failing than why an error is being logged.
Agreed, there isn't unlimited time to perform testing. Sometimes you also know when you're writing tests that will never help find a bug, and that's a waste of time.
I always focus on integration testing. If I have time I'll write unit tests, but there needs to be a good reason. I'll unit test an area that has proven to be buggy even with integration tests, but this is rare.
Finally, some code is just really complex and/or critical. These are good candidates for unit tests.
The one place I still use unit tests is around calculation algorithms where you want to push a lot of different data through. You could do it with an integration test, but the test run overhead of say 100 parameterized integration tests is far too high. In these cases the algorithm will still be exercised by an integration test, but the nitty-gritty will be done in a unit test.
Integration tests are great, but the very real downside is that they tend to be much more complex to write than unit tests, and can in some cases require setup and tear down overhead that makes it impractical to lean on them for the majority of testing logic.
Unit tests for each unit of functionality, and a small number of integration tests to validate all works well together.
Again, I think it comes down to tooling. Our app is basically a document management system with quite complex business rules. I can write a test that takes a document from inception to completion in about 15 LoC because we've put time into our test framework to provide sensible abstractions over our API.
Setup and teardown can be an issue, but we've found a good balance. We spin up a clean DB for each test in less than a second. Running out full suite does take a couple of hours, but that's what CI servers are for.
I think unit tests can be divided into two camps as well: there are the trivial ones that are very close-coupled to the implementation, like, check if my ad hoc stack implementation will actually add the element and return sensible outputs. Something like this can easily be a private method as well, and I think they make sense to include right next to the function itself. Some languages allow these reality checks right in the documentation itself - and I don’t see who would it hurt. They are closely coupled to the implementation so sure, they will have to be rewritten with it, but that’s completely fine.
The usually called unit tests indeed should not directly test implementation details, like private methods, but then we have to find a different name for the former because I do think they are useful sometimes
Adding the single biggest reason I have encountered: Because the overall functionality is too complex to test through the API alone, so the alternative is to extract part of the "private" functionality to a separate class that would then be public itself, making functionality public that shouldn't be. By that, the discussion about "testing private methods" ignores the elephant in the room, which is that the language used doesn't have a rich concept of public vs. private in the first place.
As evidence for that I'd like to mention that I never had that problem in Rust, because I can simply extract the private functionality into helper functions within a private sub-module and unit-test them there, without that functionality becoming public to the rest of the code.
If your class is doing so many things that you can't reasonably test everything but through the private methods, how about extracting a helper for those things which has testable public (or package private, if you have to) methods!? Did I just sneakily bypass the rules, or make the code better overall? I don't know anymore...
> If your class is doing so many things that you can't reasonably test everything but through the private methods, how about extracting a helper for those things which has testable public
When your solution to not wanting to write tests for important behavior is to trick yourself into wanting to by adding extra layers of code for no other reason, maybe the rule stopping you from doing it was wrong.
I acknowledged right away that it was a hack. And if it wasn't clear, by that I meant that the private method rule shouldn't be hard. It's can still be a smell, though, but I'm trying to be pragmatic.
It's not about that imo. It's about having tests that are limited in scope as much as possible, so if they fail it's much easier to understand the problem.
Ideally tests should be run in a specific order, from most low level to most high level. So tiny basic functions first, then functions that use those functions, then functions that use THOSE functions, etc until you have end-to-end tests.
Unfortunately I know of no testing system that has such a hierarchy.
In my experience refactoring and making things public is an entirely reasonable thing to do!
I often find that when I have a very complicated structure with lots of behavior, there’s some other cohesive unit of functionality living inside. That part is often private, but extracting it into a separate structure with public interfaces makes complete sense. From the perspective of consumers of the original class, all of this is still implementation detail that can’t be directly accessed. But there’s nothing wrong about accessing that underlying bit directly through its own public interface.
Private methods can benefit from knowledge about the class they live in, and be only good enough for their purpose in that class. When elevated to public methods, you may want them to be more defensive than when they were private, which is an investment that doesn't pay off in itself. People may also be tempted to reuse your public methods now, even if they were never intended for that. Java has package-private for that (I guess). I'm not a fan of package-private, but then again, I'm writing small micro-services and considering the code inside this service "friendly" to my class, i.e. not abusing public methods without looking inside them to see if they're up for the job.
That's not really what I'm talking about at all. Private methods are often useful for handling internal details. But often when you have a lot of private methods, you're bundling too much functionality into one object. I have found that it is very common that I get significant improvements in readability, testability, and comprehensibility when I decompose a large structure with a lot of private methods into a few smaller structures that operate on each others' public API.
Why you should test private methods: To make sure they work correctly. This may be significantly easier than testing the public methods that use them.
Why you shouldn't test private methods: Because the implementation might change, obsoleting the tests. But everything might change (except the end-user requirements, haha), so by this argument you shouldn't test anything except what is in the spec/what the user will see.
The only benefit of not testing private methods, is the same benefit that not testing anything else brings.
This is all there is to it. The dogma around not testing private methods is entirely unconvincing. I have some very complex private methods that it’s just easier to write some thorough unit tests for, and it guarantees the contract my public methods rely on. I really do not see why that is controversial.
If you don't have time to write good tests, maybe consider not writing any tests.
I've worked for many a startup where we didn't have any tests because we rewrote everything every few months.
And that was ok because everyone was happy to have a lot of shit written fast.
I guess maybe what I'm trying to say is that shitty tests are sometimes worse than no tests.
I don't agree that private method tests are shitty.
However, I do agree that maybe there is a time during which it's OK to just not have tests. If you are just spitballing it, prototyping, code jamming, maybe even blasting out an MVP for alpha or beta testers. Sure.
Part of mastering a craft is making reasoned decisions about breaking rules.
I can see a process of triaging and deciding that some code isn't worth testing (yet), but the broader idea of "this is just a prototype", I'd like to push back on, unless you work in the rare place where they actually start over after the prototype. "Blasting out an MVP," you say? That should be production code then, with production-grade testing. I too work for a startup, and well-tested code allows us to release often and with confidence. I am humbled every day by unit tests slapping my wrist when I attempt something stupid.
> unless you work in the rare place where they actually start over after the prototype.
Every line of code will last some amount of time. Some lines will be maintained in production for decades. And others will be thrown out in a few weeks. For example, if I’m mocking up a UI, the css I write won’t outlast the mock-up. Or doing a game jam. Or prototyping a different way to structure some code.
Unit tests are useful for long lived code but they slow down your capacity to do scrappy iterations. Whether you need to prioritise short term velocity or long term reliability depends on the needs of the project you’re working on.
Code just tends to live longer than developers often anticipate. They put more trust in their organizations than is warranted about how much time they will be given later to go back and clean things up. (This probably happens because the original work is over-estimated and then the project is delivered under in a crunch, instead of having some breathing time to clean things up.) But again, this is all situational, you may work in different circumstances than I ever have.
> Code alone can only tell you the "what." If the "why" is not obvious, comment it.
But isn't that the common view? I think most programmers would agree that the "why" deserves documenting. The problem is that in the heat of the moment, most forget.
Just to intensify the sacrilege here: I’ve been happy testing private behaviour with tests directly alongside the code. This way you can easily sweep away the test code if you change the implementation, and it serves as decent documentation for complex implementation details. The tests obviously disappear in production builds either way.
In Clojure, for example, tests are defined by a macro that decorates a function with certain metadata marking it as a unit test. So they can live and be run from anywhere, but get stripped out of proper builds. You can also go one further, and wrap a function definition in a `with-test` block, so any assertions you make against the function are kept directly alongside its definition, which is a nice form of documentation.
What this means is that you can have tests for implementation details, that are cordoned off from your main test suite (which defines your code's real requirements and public API). The benefit is that if you change implementations and suddenly a function is no longer needed, you can just delete the test code right there in the same file, so refactoring is very slightly easier and more obvious. You can also decide whether they should be run during your build process or just at dev time.
When you read software engineering books like Clean Code, they are generally heavily in favor of writing good comments.
When you read programming books, like "Learn Language XYZ" books, there are generally no comments whatsoever because there is no need. Which makes sense in the scope of such books, but I think it accidentally sets a precedent for eschewing comments in the minds of many.
But.
Out in the "real world?"
I'd say 90-95% of coders don't comment a damn thing. At least that has been my experience working in the industry since the 90s.
Some coders are in the obnoxious and toxic "code should be self-explanatory" camp, which is objectively dumb. Code can't explain intent, such as business rules or weird shit you're doing to work around weird shit in external libraries or APIs or hardware bugs.
A greater number of coders are of the mindset that inline comments are bad and that things should be explained in the commit message and/or pull request. This is more noble, but I think it is not nearly as practical as inline comments for a variety of reasons.
Many coders also believe that comments make code harder to read. I find this baffling and dumb. Get a bigger monitor or get an editor that lets you collapse/hide comments with a keystroke.
Many of his followers perhaps, but at least in the Clean Code book there is a long chapter on comments, where about half is dedicated to "Good Comments" that include many reasons for when and how to comment code. Yes, it has a section titled "Comments Do Not Make Up for Bad Code"... but he also writes about how there "are certainly times when code makes a poor vehicle for explanation" and that some comments are "necessary or beneficial" like "Informative Comments" or "Explanation of Intent" etc. That chapter is one of the few I remember as somewhat useful in the book (I am not much of a follower or fan... but I did enjoy his Clean Architecture book).
I have a tendency to comment code that isn’t optimal and will need to be refactored for performance at a later date.
Not sure how many times I’ve been looking into a new performance issue as we grow, and come across one of my own comments.
Sort of falls into the Good/Fast/Cheap trifecta. When you have to do the Fast/Cheap bit, note when you know it’s not optimal so you can identify it quickly when you or someone else comes back to it.
I don’t think you shouldn’t comment your code, but in my experience, good code doesn’t need much, if any, comments.
This is because I define “good code” as simple code. If code gets to the point that it needs to be explained with comments, then there’s a good chance there is opportunity for simplification.
In other words, if I can’t read code and follow what it’s doing — or why it’s doing it — then I won’t ask them to add comments, but to instead refactor/simplify the code. (This is assuming the code can’t be made more readable with simple name changes, which a lot of times is all that is needed.)
Yes, there are times when the “why”, even with simple code, needs to be explained. Comments in this case are fine. But ideally this is the exception and not the norm.
There are numerous cases where code needs comments. People who think identifier names are all you need are frankly frustrating. It forces co-workers to do reverse engineering any time they want to understand the code.
The most obvious case are API docs which includes internal APIs (means: any public method, in my codebases). Comments are much faster to read and understand than the source code, can be quickly pulled up by using the right hotkey in your IDE, and can be browsed via generate API doc pages in a browser. I don't want to read your function to figure out what it does, I don't want to guess how I'm supposed to use it from the name. I want proper docs.
The second most obvious case is where the reason for a piece of code isn't obvious. Typically this means bug fixes or workarounds, but can also be performance optimizations. If something isn't explained by a comment then sooner or later it's going to get accidentally changed. Unit tests aren't a complete replacement because you don't necessarily know what tests are for what lines of code, and often it's hard to unit test an obscure bug fix anyway (e.g. workarounds for OS level problems).
I’ve never understood why anyone would choose to debug by printing and logging if they have a good debugger available to them.
My day job is mostly PHP (yeah yeah, I know) and the general attitude towards Xdebug in the community seems to lie somewhere between apathy and active avoidance.
To me, a debugger is such an invaluable tool not only for identification of bugs but also for familiarising oneself with new code. The value of being able to not only trace execution from start to finish but also to modify state in a running program is immense, and gives me a small taste of what Lisp folks rave about.
Littering print statements everywhere feels like banging rocks together when there’s a perfectly good lighter in your pocket.
It depends on what you're doing. I find console statements helpful when testing out a new function that you don't necessarily expect to misbehave so I check the internal state is correct without having to stop the code.
Debuggers provide much more information, but stopping and stepping through the code is much more time intensive than printing out variables at various points and running the function and doing a quick double check that they're correct.
It also allows you to check the state of the program earlier for problems that may only emerge after the fact. Instead of having to stop what I'm doing and retry and step through with the debugger, I can just look back at the log and see what went wrong where.
I think both debug logs and debuggers have valuable use cases.
Tracing debuggers give you the best of both worlds. I've recently started using Flow-storm [0], by @jpmonettas), and it's been quite transformative. You can still easily see the values flowing through your system (better than just "prints"), and it can handle multi-threaded / async scenarios quite nicely. You don't need to manually step through code, you can just "see" your data flow, and when you have loops or some other form of iteration, you can see the data for each pass. Coupling this with a good data visualization tool (such as Portal [1]) really feels like magic. I've been doing Clojure for quite a few years now, and was very happy with my plain REPL-driven workflow, but this is way better.
Or God forbid those Next.js apps where the same line of code is ran on the server, streamed to the client, and then ran again on the client via rehydration. 3 different potential sources of failure and you have to debug them simultaneously.
There shouldn't be a need to connect to another service during debugging except maybe to validate that an external service stands by its published contract. A debugger (nor printf) does not seem like the right tool for that job, though.
If you're not sure which service is causing the error, combing a log output from the two services is much easier than debugging each service separately, is my point.
1. How does this really differ from debugging both services at the same time and attaching logpoints?
2. Microservices is a team separation technique, allowing different team to live in isolation, sharing only API documentation. It's the same as regular web services, except within a single organization instead of across organizations. If you happen to have access to the code (not a guarantee!), are you really going to be able to jump into a codebase you're not familiar with to start adding print lines with more expediency than other possible approaches?
Re 2: tell that to the company I used to work with. Management heard microservices are good? Then we should have nano services. They mandated that we split things up super small, by functionality, even if there was only one team working on that.
Yes, that's right. Teams not needing to communicate, beyond API contracts, allows them to scale the service they provide. When sharing code in a monolith then you have to meetings when you want to make changes that impact other developers, and that will soon take your whole day when there are thousands of workers, leaving no time to do anything. That leaves service that cannot scale.
More simply, it takes how the macro economy scales software (and, really, most everything) and tries to apply the same processes in the micro.
This just means that debuggers have to catch up. Anything that runs on a CPU is also debuggable, and if it's too much hassle, the debuggers need to improve. Debuggers are much more than just stepping through code and setting breakpoints, they are (or should be) "interactive program state explorers and visualizers".
(IMHO no programming language should add features which make interactive debugging harder, it's never worth it in the long run)
There are various kinds of debugger. For investigating systems like this I think you want a trace / time travel based debugger, like eg FlowStorm in the Clojure world.
I use both, they have different purposes. I find printf-style debugging to be much faster for bisecting the control flow, and the vast majority of bugs I see are of this type.
Debuggers tend to be more useful in cases of corrupted state that isn't the result of broken control flow logic. These are much rarer, usually the product of something like broken bit-twiddling logic. And these days that kind of code is rarely written manually. When I write a non-trivial data container, I usually provide a standard public method that does a deep inspection of the integrity and consistency of the internal state. These are much too expensive to use as a typical assertion, but turning this on usually isolates the problem so well that a debugger isn't required to figure out what went wrong.
I used to use debuggers a lot more than I do now because the nature of the typical bug has changed. The types of bugs are also much less diverse than they used to be.
One of the great UX features of printing is that it is a quick way to have an auditable form of state over time. I look at states 1-n, compare and contrast, which my debuggers don't make as easy.
I'd add many people don't work in languages with good debuggers. I work in TS and the debugging async node code is an absolutely garbage experience in many cases.
For many nontrivial debugging tasks. You need to record a lot of past context - e.g. you can put a breakpoint when a bad decision gets made, but the data causing this decision likely accumulated over a few tens/hundreds of previous iterations - and in particular one bad iteration that you don't (yet) know how to characterize. In such a case, setting up watches and pressing F5 a hundred times to get to bad iteration is inferior to adding a print/log and scrolling back to it.
It's not one or the other, most debug sessions will also include some sort of adhoc logging.
Also "modern" debuggers ("modern" as in "this century") offer a lot more than just simple on/off breakpoints: conditional breakpoints, data read/write breakpoints (e.g. who wrote this broken piece of data), "tracepoints" which don't stop execution but instead log out a message or run some scripted action, etc etc...
A good debugger is essentially an interactive program state explorer and visualizer, ideally it would be fully integrated with the actual development process (e.g. edit, compile, run, break somewhere to inspect program state, tweak some code while program is suspended, rewind a program state a bit and continue running with the modified code, rinse, repeat...).
The problem is that the debugger conditional logic has to be written outside the local context of the code, and probably in a different language to boot. For example, in Xcode, you’re entering commands into a tiny box in the breakpoint popup that won’t actually tell you if you got the syntax wrong or not.
You don't need to litter print statements, you can litter log statements :D
I can't even remember when was the last time I was on a project where debugging would've been even a bit doable. Maybe, just maybe, during development when it's running on my computer.
But it worked on my computer, that's why it ended up in production =)
With proper logging (or just print statements) I can go in after the fact and narrow down what happened when it didn't work in a specific situation.
They also let you troubleshoot a problem after it has happened. Logs can give information about the trigger, which the debugger can't do as it requires you to known what inputs caused the issue.
Can you please explain how does debugger can help in cases where you can't wait too long on a breakpoint because some thread or service waiting on it will fail? I do this many times at work in Java with intellij. I do fine with breakpoints most of the times but sometimes print statements is the only way to trace the thing.
From your comment I feel like I am missing better ways to debug.
Thing that annoys me is I know that debuggers could do a quick grab and dash.
Either stop the program snag the state of a variable and go. Or set a sample point where when it gets hit the debugger takes a sample of local state and continues. It aught to be able to do that within a few hundred clock cycles.
Myself I use error/debug printf, a command line interface, debugger, and sometimes an oscilloscope.
With error/debug printf I leave them in and just turn them on/off with a compiler switch. Over time they tend to evolve into something that provides good info on what's going on.
The command line interface allows me to inspect the program state on the fly and run tests on the fly.
The debugger allows me to do random look sees of what the program state is and follow the call tree.
Oscilloscope allows me to inspect/debug/verify timing issues.
My suggestion to you is implement a command line interface to you program so you can interact and inspect your program while it's running. It's really not hard at all. Implement commands as a table. { "command", handler, extra_data}
I don't know how this can be used in a very old and large codebase where no single person knows everything. There is too much state and unknowns at any given time. IDE with breakpoints does help explore the state through stack.
Log breakpoints are still kind of print debugging. I do add log breakpoints, even add conditionals on them, they however become a huge performance bottleneck in places and best way is to write a print statement in the code itself.
Print debugging can be much faster than using a debugger if you can figure out the problem and fix it on the first try. For more complex issues it makes sense to switch to a debugger.
You really shouldn't have to "switch to a debugger" as if it were a separate tool, but instead have it integrated into your development environment (e.g. "run" or "step into" should both be just key presses, and every "run" automatically runs inside the debugger so that you can pause and inspect program state at any time).
Sure, you shouldn't have to, but sometimes you're developing in an environment where adding a print statement + restarting + reproducing is faster than waiting for your IDE debugger to download whatever state information it needs (or whatever it's doing) l to render a prompt. For example, Xcode takes an eon to show a lldb prompt when you hit a breakpoint. Printing is instant every time.
Further, there are cases where variables are optimized out of builds so you have to add code to ensure they stick around (in which case you may as well print). Again, this applies to iOS development. Maybe others.
I don't understand how that is possible. I would say the exact reverse. With print debugging you have to write the prints. With a debugger you just run the code. Especially with conditional breakpoints...
Takes me considerably less time to dump a bunch of prints than write (multiple) working conditional breakpoints. That said it's very problem dependent whether I use a debugger or logging (often a mix of both) and rr has considerably reduced some uses of logging for me.
> I've never understood why anyone would choose to debug by printing and logging if they have a good debugger available to them.
Because logging provides a different experience and can be sometimes much faster. I can run the code once and then analyze all the collected information later. The log allows me to go back and forth very fast, or very quickly see that e.g. a 150th iteration went sideways even if I don't exactly know what to expect. Another area where logging is tremendously helpful is debugging concurrency related bugs.
Yep. Most debuggers also make it very hard to go backwards in time. Logs can be read in any order.
There’s a neat trick I like to do when I’m trying to triage a bug that only shows up with some input data. I’ll run the passing test and the failing test with detailed logging, then diff the logs to understand how the code is behaving differently in each case.
A jr programmer once asked how I know that N lines do what I expect without writing a test every 10 minutes. Logging is a form of unit testing. Logging can be thought of as unit tests for your specific context, decoupled from the logic (almost) entirely.
I have an expectation of state, the execution can be checked against that expectation via observation of the output (which usually has serialized state). It's not a fancy True False or checkmark, but it's an assertion (which constitutes a test of an expected state) nonetheless.
A log statement isn't a unit test in the same way that apples aren't trees. A unit test is a discrete, runnable program that asserts some state about your program to be true, whereas a log is just some subsystem of your overall program that prints some output with no pass/fail condition.
You may have a unit test that makes assertions about logs though? I would definitely expect somebody that isn't a junior programmer to understand this.
> A log statement isn't a unit test in the same way that apples aren't trees
I don't think your explanation is compelling.
> a log is just some subsystem of your overall program that prints some output with no pass/fail condition.
The pass fail is interpreted. A unit test result is interpreted, be it with a big green/red or an exception or any other way you wish to express it.
Running something manually and checking the logs in an environment, when there is a reported issue, is modus operandi of every developer today. Did A happen and B happen and what did C say before exit? A looks good (because it has an expected state), etc.
> I would definitely expect somebody that isn't a junior programmer to understand this.
I didn't say logs were a replacement for unit tests (that's a previous poster's strawman). Regardless, they are a form of testing, because we use them to observe state after execution. Manual testing is testing, manual testing and observation of logs, is testing.
You explicitly said logging is a form of "unit test" which has a very specific meaning and a log is not it. Observing a log can obviously be part of an assertion in a test be it manual or whatever, but an assertion is not a unit test.
> A unit test is a discrete, runnable program that asserts some state about your program to be true
I would say "Observe the state of an execution stack". Tomato tomato.
> which has a very specific meaning
Unit testing is a methodology, not a specific thing.
How you observe it, be it through a pass fail indicator or a "This looks right", is immaterial. Splitting hairs on the output methodology does not change the utility.
Unit test frameworks are the modern, convenient, methodology to control bubbling up an output, when multiple assertions fail. This is what observing the red (suite failed) or green (suite succeeded) is.
Executing a suite requires a manual interaction (ie > run suite or > make build -> runs suite or > edit cron -> runs suite). In a way, all testing is still manual, but test suites allow us to scale testing, which logs do not (1million log files is impractical to use as testing output). That's why unit test frameworks are a good thing.
Eyeballing a log is often enough to convince myself that some code is probably correct. It’s not as thorough as a unit test, but better than nothing if you weren’t planning on writing a test anyway.
This can also be a great way to write your unit tests. Once you like what your code does in some situation, you can copy paste the input and expected output into a unit test to make sure you don’t break that correct behaviour later.
> My day job is mostly PHP (yeah yeah, I know) and the general attitude towards Xdebug in the community seems to lie somewhere between apathy and active avoidance.
From my experience, simply having xdebug enabled comes with a significant performance hit, which is why I generally avoid it unless I really need it.
>> I used to think that the names of the classes, methods and variables should be enough to understand what the program does. No comments should be needed. Over the years I have realized that some comments are needed and useful.
Comments are needed where there is more to the code than just reading it. Where there is some external reason WHY. Where it is written in a specific way for non-obvious reasons. Where there are pitfalls and dangers in changing the code. Where this specific approach is the result of fixing some problem and if you change it then you might be reintroducing a problem.
There's lots of reasons to comment your code, but mostly I think code should be the documentation.
The fewer comments the better, because then developers who come later will see comments and thing "this must be important because there is a comment here". Too many comments dilutes the value of comments.
When it really matters I start my comment with a couple of lines like this:
// LISTEN UP!!!
// LISTEN UP!!!
I have to say, it seems strange that the author EVER thought that no comments should ever be needed - that seems like a strange and dogmatic conclusion to have come to.
> it seems strange that the author EVER thought that no comments should ever be needed
That was a trend many years ago when people started seeing how comments are often out-of-date, sometimes been copied pasted around so many times it's not even saying anything useful about the code next to it, and sometimes the author is just a bad writer and makes it more confusing than the code itself. So people became pissed off and kind of decided that the code needs to be clear enough to be self-documenting: short functions with verbose names, good variable names etc. making comments unnecessary.
I do agree with that, but IMHO there's still a place for comments: as many others are saying: when you need to document why something has been written that way, not just what (which the code should be able to tell by itself)... and I believe OP is also claiming that, and you appear to be missing this context.
> it seems strange that the author EVER thought that no comments should ever be needed - that seems like a strange and dogmatic conclusion to have come to.
Distressingly common, though. I'd say most of my last few positions have been at places where "self-documenting code" was the mantra even though it was blatantly clear that the code and what it was doing was far too complex for that to work.
That gets me thinking, if the code is obvious enough it doesn’t need comments, is it useful code? Like in an ideal world, all the “how” is transparent and we’re left only with the juicy bits, that explain / define business needs.
I’ve noticed that different language have various ideas about this - like
Closure would be close to a fetish to build abstractions so that the actual code that does “stuff” just explains the business process, and the rest are just libraries.
Or golang goes in the other extreme where there it is idiomatic to have as little abstraction as possible, writing up everything as you go.
Haven’t coded much in either so this is just a beginner’s observation.
Programming language generations may be relevant here if you are interested. Prolog for example can be called a 5th generation PL, which means that you state what you want instead of how you want it achieved. Wikipedia lists SQL at the 4th gen, though I think the former is just as true for SQL as well - you only state that you want this and that row/field, and the actual algorithm is up to the query optimizer.
> it seems strange that the author EVER thought that no comments should ever be needed - that seems like a strange and dogmatic conclusion to have come to.
But just before that you say:
> There's lots of reasons to comment your code, but mostly I think code should be the documentation.
Exactly that's why it's not at all strange. As you yourself write, code should be the documentation. Need to add comments => your code is unclear, and unclear code => bad code, and so instead of writing comments your time is better spent improving the code.
The reality of course is that things need to be done and there's no time to fix all layers and have perfectly organized and readable code and therefore clarifying comments are needed.
So no need to shame the author, it's just a typical progression from youthful maximalism to more mature shades of grey thinking.
I think the "never use comments" trend came from folks who'd realised "comments lie, code never does".
E.g. developers might forget to update comments when they fix a bug.
There are also the "// set b to 0" brigade, which is an exaggeration but makes the point that commenting the painfully obvious is a trap too many fall into.
I think there was also an argument that methods called "setWidgetsToMaximumSoThatStartOfDayChecksWorkOnWeekends()" were a Good Thing, and definitively Self Documenting, a viewpoint which seems (thankfully) to be losing popularity.
I wish that my programming languages had a "public for testing only" scope. Despite the rigorous arguments of the Properly Factor Your Public API crew, I still find myself in real-world situations with functions that should not reasonably be used outside of the module, but that I still find valuable to cover with unit tests.
@VisibleForTesting is mostly documentation; you can just as easily use package private without an annotation. (Though yeah, I think it's good to use the annotation anyway.)
yeah, that one stood out to me too. i would be tempted to make the stronger statement that it is always strictly better to have your private code unit tested, and it is only the limitations of languages and test frameworks that make people reach for ways to test them through the public API.
Allow me to even more strongly disagree. I've not yet seen a situation where strongly-tested private code would have helped, although I routinely come across cases where too many tests get directly in the way of a refactor or bugfix that _would_ improve the program.
These days, I'm of the opinion that you should test at the highest level possible (if testing a web service, for example, at the level of the actual public API to the service), thoroughly define the behaviours expected for those APIs, and leave the insides of the service entirely untested, unless you have _very_ good reason to. (Critical code, payment code, hard-to-test code, etc.) This ensures you have the important behaviours tested, but leaves you free to modify implementation details as necessary without needing to alter or rewrite entire swathes of test code that suddenly starts failing because you decided to break up or consolidate an inner method.
contrariwise, i have never come across a case where having tests for private functions/methods has hindered refactoring or bugfixing. that feels more like undue coupling than undue testing to me. if i have to alter or delete swaths of code i just delete the associated tests as well, and write new tests for the new implementation.
perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code.
i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.
Too many unit tests can slow down refactoring. But it can still be worthwhile.
Eg: I’ve done some deep algorithmic work on CRDTs lately and my code has a lot of internal parts which are all quite fiddly to implement correctly. For example, I’m using a custom btree with some unusual features as an internal data structure. Btrees are famously difficult to implement correctly, so my btree has both unit tests and fuzz tests to make sure it does what I expect. Having that test suite makes integration tests easier to triage, since I know any failures in my integration tests probably don’t come from the btree. And some btree code paths are probably rarely or never executed by my integration tests. But I still want that code to be correct. Testing it in isolation is the best way.
They’re a lot of other small pieces that I test like this - like saving & loading code, my graph traversal utility code (for comparing versions semantically) and so on. Low level unit testing can find bugs in utility modules while you write the module itself, not later (when you start using the module in some other code).
Agree. The core downside to this approach is pinpointing failure points. A large integration test failing can have many causes. In the same vein, technically, detailed unit tests can immediately discover these. However, in reality, having such a tight, well defined, correct web of unit tests such that any failure can be immediately traced to individual methods is unlikely and an immense amount of work.
An isolated failing integration test implies a better unit test could exist somewhere. But as you point out, you still probably came out ahead by not writing _all_ the possible unit tests you could have.
I go even further - for the browser level tests, avoid using “testid” or css classes or anything that is “implementation”, but rely in your test on solely things that the user can read / interact with.
So don’t “Press button with id “generate”, but the button that says “save” inside the content element titled “generation”.
This way any refactoring work would not require test changes (as it should) and any change of the visible ui / workflow to the user would require an adjustment to the test.
This is a style that I learned from ruby’s integration testing framework “capybara”, and have been replicating it wherever I can since.
A nice bonus is that if you switch rendering technologies, you can reuse the tests (like react native for example).
But on the other hand: if it's worthwhile testing your private methods, shouldn't they maybe be refactored to a public/package-private scope, so they can be reused?
If you disagree, maybe an illustrative example would help. I couldn't think of one where I want to test a private method in detail that is not worth exposing.
A common case for this is very simple APIs with very complex internals that e.g. dynamically switch behaviors or algorithms in certain contexts. The only way to test all the major code paths through the simple API is to hardcode implementation detail into your unit tests to induce all the various switching contexts indirectly. It makes unit tests quite brittle. At the same time, these internals are definitely not public APIs and unusable as such.
> The only way to test all the major code paths through the simple API is to hardcode implementation detail into your unit tests to induce all the various switching contexts indirectly.
Is the "only way" not property-based testing (and maybe fuzz testing)? If that doesn't get you there, it is likely that the API is poorly designed.
You are assuming the function being tested is more trivial than it actually is. Often these are too small (code-wise) to even have private APIs, the internal behavior of the function is simply and necessarily too complex to unit test in the traditional way.
Some function behavior is intentionally and intrinsically tied to temporal access patterns in the API usage. Unless you can simulate a broad cross-section of real-world runtime API access timing patterns with your testing framework, you won't test all of the code paths. You often see this test problem with scheduler-like functions, where the correct function behavior varies based on internal resource pressures that are an interaction between temporal access patterns of the API and the runtime environment. It is a single function, self-contained, and quite simple, maybe not more than 100 LoC, but test environments are so sterile that usually only a single code path is actually used no matter what you throw at the API.
Some functions have critical code paths that dynamically switch strategies to mitigate when certain types of contention or resource starvation conditions are detected internally. These can be extremely rare cases across the set of possible inputs, such that fuzzing is unlikely to trigger them, or the set of inputs that can trigger them is dependent on exogenous environmental details e.g. the machine where the test is run.
As a fun side-effect, sometimes these functions do not have deterministic result. Getting the same result out of the function all the time does not imply correctness, you also have to know why you got the result you did.
All of the above does not apply to writing business logic in Java or similar. But for high-performance and/or high-reliability systems software, these cases come up often enough that it is a well-understood testing problem. Even if you expose all of the internal implementation detail to the unit tests it is not always possible to reliably trigger all the code paths from an arbitrary test absent purpose-built test tooling.
This is always the theory: "well pull that out into a separate class with a public API". But then those classes end up only being used in one place (the class it was extracted from), with a totally unnecessary increase in public API surface area. Both the extra verbosity and the pointless increase in API surface area are worse than just writing unit tests against private functions IMO.
here's an example - i'm currently working through porting the blossom algorithm [http://jorisvr.nl/article/maximum-matching] to elixir. the algorithm is complex and has a lot of subparts, but the public interface to it is extremely simple. i feel like it's not just useful but almost essential to test a lot of the little parts in isolation so i can make sure they are working correctly.
i am currently marking all functions as public for simplicity, but i feel like it's a failing of elixir and/or its test framework that i cannot say "private, but an associated test module should be able to see them", and once the code is done i will be exploring some of the third-party solutions people have come up with to hack around the issue. (the other option, of course, is to have a new module that just contains a couple of public methods, and regard the entire implementation module as a private module with public functions)
Go comes with this kind of “public” and “private” separation. Only the private tests can access private functions. It also serves to differentiate which tests are for API documentation, which the API must conform to forevermore, and which were used to help with development and can be considered throwaway.
I'm not sure why all testing frameworks don't have this.
A lot of the time _private_ methods on one class end up making more sense as public methods on a helper class of some kind (or public methods on other classes).
One issue that leads to private methods in Java is the lack of any way to extend the set of operations on built-in classes: a problem that Kotlin and C# solve with extension methods.
Extension methods are basically just static methods in a helper class that takes the to be extended class as their first argument, and a bit of syntactic sugar on top. That is absolutely imitable in Java.
This exists (in kind of a hacky way) in C# with the ExternalsVisibleTo attribute. It exposes methods and classes only to specified assemblies (like your unit test assemblies).
I put things like that in a separate namespace. If my namespace is Foobar I'll have a Foobar.Impl namespace.
One of the lessons I've learned is that very little needs to be private, and there's a lot of advantage of not having things private.
Putting that in a clearly defined namespace allows access when needed, for testing or workarounds, while conveying the needed "if you use this you're on your own".
Go has this. Python has this (_ access). package-private allows you to write tests if the tests are in the same namespace (Java langs).
Class Access Modifiers were for controlling property and method inheritance, which has been bastardized into "CAMs tell you the API", because it's convenient. An object's API can be defined by an interface (following traditional OOP conventions), allowing CAMs to be used for their original orthogonal purpose. In practice, developers don't want to make an interface when they can slap a CAM on it and say "good enough". Some language don't even support interfaces, pushing the practice. So here we are with the wrong abstraction.
Ruby also has this. e.g. you can use send to call private methods during testing (and then use static analysis like rubocop to ensure only public_send is used in production).
You also can simulate it in JavaScript by using a closure and a separate package for your "private" functions. By mixing the "private" code into the closure you can make it private in usage but still available to unit tests.
> I wish that my programming languages had a "public for testing only" scope.
> Python has this (_ access)
Bypassing CAMs is useful for testing. Python has a mechanism, which is used for that^...regardless of the intent of the feature.
^Underscores are a partially enforced convention for private methods. If your import stripped your access to an attribute, you can still access via __getattribute__() which reaches into the hierarchy. There are other tricks as well. You might call this a special method lookup.
I still don't understand what you're talking about.
> If your import stripped your access to an attribute
Python doesn't have class access modifiers, and in particular import does not strip access to attributes in Python. There are two ways Python modifies behaviour in the presence of leading underscores:
1. If there is a module attribute (i.e. a variable in a module) starting with an underscore, it will not be imported when doing `import * from some_module` -- which is not recommended anyway. Importing `some_module` directly will give normal access to the attribute, as `some_module._foo`.
2. If a class or instance attribute's name begins with a double underscore, it will not be accessible directly; however its name is simply mungled at compile time:
> class Foo:
> __foobar = "__foobar"
>
> def __init__(self):
> self.__bar = "__bar"
> self.__baz__ = "__baz__"
>
> Foo.__foobar
AttributeError: 'Foo' object has no attribute '__foobar'
> foo = Foo()
> foo.__bar
AttributeError: 'Foo' object has no attribute '__bar'
> Foo._Foo__foobar
'__foobar'
> foo._Foo__bar
'__bar'
> # Note that does not hold for attributes both starting and ending in double underscores
> # (so-called 'dunder' attributes):
foo.__baz__
'__baz__'
You are correct that underscores are a convention, but in Python one simply cannot count on the kind of access control that exists in other languages, especially when writing code that is supposed to be executed in unknown contexts (i.e. a library). When writing tests, one presumably has full control over both the code and the execution context, so this kinds of tricks can be used relatively safely; that being said, test runners like pytest tend to provide their own abstractions and tools around those tricks anyway.
_single_leading_underscore: weak “internal use” indicator. E.g. from M import * does not import objects whose names start with an underscore. - https://peps.python.org/pep-0008/
Your first point is something I mentioned in my comment (although admittedly I mixed the order of the command). In general it's a bad idea to import everything from a module into the global namespace; it's recommended to keep it under some namespace for context:
I.e., assuming my_module has function foo, instead of:
It would be interesting to know how long the author has been developing software.
For me, this is the kind of stuff I questioned in my first few years. After I productionalized a few real world systems for a business, the whole “dev street cred” thing lost it’s appeal. It wasn’t about some imaginary “dev purity” thing anymore, it was about being efficient and making sure I was contributing to the bottom line.
Maybe. I have been developing for over 30 years. I still use emacs, use logs and printf for debugging, and have never even looked at ChatGPT. I'm willing to admit I may be set in my ways but I get my work done.
Nearly 40 years here — I exclusively use IDEs and debuggers and have done so since the early 1990s.
Computers aren’t digital pencil and paper. They’re levers for the mind. If the 3 GHz processor just sits there idling, waiting patiently to copy some bytes from one buffer to another, it’s being wasted. It could be checking the syntax, looking up documentation, or chasing memory references in the debugger so I don’t have to.
I’ve used both debuggers and print/log statements. More recently, I’ve learned the value of even seeing if I’m in the right part of the code by inserting 1 / 0 (or any way to raise an exception if 1 / 0 doesn’t blow things up). If I can’t even get a bad arithmetic error, I don’t need to worry about the rest of it!
Programmers had a very nice thing called "web search". Then advertising and spam ruined it.
From my point of view, ChatGPT is simply an anti-spam algorithm which is finally restoring "web search" back to being useful.
The problem is that there is no money in that. So, everybody is trying to apply all the ML/LLM/ChatGPT stuff to absolutely anything in the hopes of making some money.
> From my point of view, ChatGPT is simply an anti-spam algorithm which is finally restoring "web search" back to being useful.
For now, I imagine that advertisers are currently trying to find some way to adapt their platforms to have an LLM spit out their product via some paid prompt injection that a publisher can sell them.
Bing's Chatbot already does this. Ask it for recommendations on something likely to have advertisers bidding on it, like the best mattress for a side sleeper, the space heater for (X room size), etc.
Good point. They’re all just tools and patterns. The actual tools used _could_ save some time, but experience plus the capacity to reason about software is a bigger deal.
The article just made me think of some devs that I’ve worked with that were obsessed with particular tooling (insert comment about a hammer and everything being a nail).
I do keep an eye on emacs. It turned into mainly my org-mode editor, but the v29 article that hit the HN front page recently has me curious about trying more dev work with it.
It might be possible to do anything with a chisel that you can do with a CNC machine, but that doesn't in any way mean either choice is as reasonable as the other.
I just can't see a genuine reason to not use a real IDE and debugger. You can even fall back on the same tools you had before with them! Using a debugger doesn't mean you can't also use print statements, but if I could only have one I'd pick the debugger every time.
Overall a pretty good article, however I do feel like the author misses the mark on a couple points.
> Unit testing private methods.
As someone else pointed out this leads to accidental testing. I'm not a test zealot, I think 100% coverage is a fool's errand, and I think TDD is an abomination, but well-structured, well-thought-out tests can be a game changer when used appropriately. Testing things by accident has inevitably lead me to finishing some piece of work then spending half a day or more tracking down why some test failed inexplicably.
> Using an IDE.
I think a better point here is to get really good with whatever tool you use. If you know every incantation in vim you're going to be amazingly productive. If you know every keyboard shortcut in IntelliJ you'll be as effective, but probably not much more. The person who knows vim or emacs in and out will beat the person clicking around in an IDE every day of the week.
That being said some of it is spot on in my admittedly limited experience (only about 13 years or so, in a handful of industries, never FAANG-level scale). The point about commenting problem areas in the domain has really changed my approach to comments. I don't write any comments about what the code is doing unless it's a "here be dragons, don't change $X unless you're free the rest of the week" kind of warnings. But I comment extensively why the business or regulation requires A or B to happen instead of the more straightforward C.
The ChatGPT bit as well matches my experience. For well-defined things where it's hard for ChatGPT to make the answer up, and easy for you to verify if it does (or at least low-damage), it's worth the $20/mo IMO. I tried using it to learn CDK and while I'm not sure it saved me any time, it did save me from having to trawl through AWS documentation.
I'm intrigued. TDD may bring way more pomp and circumstance than it deserves, but when you get right down to it all it says is:
1. Write a test first so that its failure state is proven.
2. Test behaviour, not implementation.
I have certainly been caught in haste writing a test after the fact, messing it up – in a way that saw it pass, and then later realized that it wasn't actually testing anything. #1 solves a real problem.
Which must mean the abomination is the behaviour testing? Do you really care about the implementation, though? Surely that can be considered throwaway? You are not going to gain anything asserting that you used bubble sort and have that assertion fail when you update your code to use quick sort.
> 1. Write a test first so that its failure state is proven.
This is useful for to me only when fixing existing code but not when writing new code. Before I write the implementation, it is obvious the test would fail. By not having to code the test first, I have more flexibility on changing the API if needed.
Is that not just the outcome of #1? Short of trying to capture many different things in a singe test, which then becomes a nightmare for future readers, a single test just isn't going to touch that many lines of code and if you get ahead of the tests by doing more while you're in there then you're back to the problem #1 tries to solve.
(1) is the distinguishing characteristic of TDD, but it is not an end in itself, it's a mere way to achieve (2). You write the test after you have the API design, but before you have the implementation, and thus supposedly decouple your test from the implementation, since you don't have the implementation yet.
This approach never sat right with me.
* It makes design iteration more costly: if you want to change the API design as you're implementing the thing, now you have to change a bunch of tests to use the new API, and that's when you don't even have the implementation to test yet! This makes you subconsciously prefer quick and dirty design patches instead of making bigger but better changes to the design. So you're statistically trading potentially better design for potentially better tests. I'd rather have the former than the latter, because it's much more insidious.
* Just like TDD encourages writing tests for behavior instead of implementation, it also encourages relying on tests as the sole arbiter of code correctness. The supposed better quality of tests is offset by over-reliance on them, including by not adding tests to check the weak points of your implementation (because you already wrote all the necessary tests before you came up with it... right?). I've worked with some TDD-loving developers who didn't even feel the need to run the code they wrote on their dev box, in the application context. Tests pass, therefore straight to production. The bugs caused by this attitude offset all other advantages of TDD.
* TDD pretty much relies on 100% unit test coverage, because since you (supposedly) don't think about what the implementation will be, you don't know in which methods the bugs are likely to be. This causes you to over-invest in writing unit tests, but writing and maintaining tests is not free. The risk of bugs is not the same in all of the code, and same for the cost of bugs. Not to mention that some of the code is better tested at a higher level with integration tests or end to end tests, but IME nobody takes TDD there, probably because it's too annoying.
* Overall, TDD seems like an overly elaborate way to proactively "gotcha" yourself with various unhappy-path edge cases before even thinking about the implementation – thinking that would, in fact, expose more of such edge cases. I find that instead of this weird backwards overhead-heavy process, the usual development process works just fine. Start with the big picture, and leave messing with small details for last. Whereas TDD requires you to bring up all the small details / edge cases upfront (otherwise it's indifferent from regular testing).
Like with most ideologies, there are some good high level principles to take away from TDD, but at the end of the day, most ideologies like this are designed to replace unreliable thinking with reliable process. And while that process may indeed reliably produce some kind of consistent output, it's not necessarily better than what you'd have without this process, or with a different process.
Unit testing private methods via public interfaces is accidental testing and leads to overly complex and brittle test code. The author’s first instincts were correct here.
This perspective is really interesting to me. I’d be interested in knowing what people find problematic with it.
I used to unit test all the components individually. But these days I test with most of the componentry wired up, only mocking the i/o layers. I end up writing fewer tests, they have been less brittle, and refactoring is easier.
Yes if you eliminate an entire class of testing infrastructure, tests become harder to write because you end up poorly implementing that infrastructure in every class.
How does this show you "issues in the code base's architecture?"
Then you have to reimplement method call count when you need to assert something was called in your interface-based mock; and reimplement differing returns based on input or call count; and maybe reimplement throwing exceptions when those are necessary to test not-so-happy paths.
In the end you'll have an implementation of the interface with quite complex logic in the methods' implementation to handle differing returns, which will couple your mock implementation to the test cases setup, or you'll reimplement a mocking framework.
I prefer to use a mocking framework to deliver stuff instead of stabbing myself to keep purity of interface-based APIs.
I don't necessarily agree, I think it's situational but I think that the dev community as a group has swung too far to the "only test public interfaces" side lately. It ignores some important realities in favour of ideological purity.
Sometimes there are well-defined processes for performing a task, and that task is performed in only one place in the system. Therefore the details of the process can be kept class private inside of the only consumer. That doesn't mean that the processes should never be tested. If the task is cumbersome to set up or runs slowly then there is good reason to test the internal parts of the process which can be tested with dozens, hundreds or even thousands of permutations of input data cheaply and efficiently. Always relying on the large-scale tests to hit every combination of inputs for a well understood subroutine can be inefficient.
You could make the argument that this well-understood process could be broken out into its own class/package/module and tested with its own public interface, but if there really is only one consumer then that's kind of a strange trade-off to make in many cases.
> You could make the argument that this well-understood process could be broken out into its own class/package/module and tested with its own public interface, but if there really is only one consumer then that's kind of a strange trade-off to make in many cases.
That's how I develop in general: a "component" does not exist because it has multiple-clients, but because it is a conceptual piece of logic that makes sense to document and test in isolation. It allows to define what is the public API of this component and what isn't. This is how software scales and stays maintainable over time IMO.
In the vast majority of cases, The Enterprise, private methods are little to no use anyway. Whenever I’m faced with updating a code base that has Draconian code coverage metrics in place I’m more apt to make a private method public than futz around trying to figure out mock magic to test the change
Whether or not it is enough depends on the context. Some cases require 100% condition and decision coverage. In these cases, it gets to be very difficult to achieve that unless you are directly testing the functions you write, not through a caller.
But even deeper, it is "accidental" because you are not testing contracts on the public APIs, you're testing the private functions through a layer higher. When the implementation changes, you have coupled your tests so tightly to the implementation that they are useless for regression testing. If not, then you actually aren't testing the private function at all, you're just hitting some parts by coincidence.
There are a number of issues. First, the fundamental unit under test is the method/function, not the entire call stack. Second, complexity starts to grow when there are code coverage gates in place.
When I see tests that are orders of magnitude larger than the units they’re testing and contain mocked references that are used 7 stack frames deep I know the author had a fundamental misunderstanding of how to write helpful and maintainable tests.
Tests freeze code and design. Often this is exactly what you want but sometimes it's not what you want at all. If your internal interfaces are unit tested then you can't refactor easily because you'll break all your tests. However, if only your external interface is unit tested then those tests actively benefit refactoring the internals.
I agree. It's also not always possible to sufficiently stimulate a private function using public methods.
He was right to change his mind about the other things.
I was also skeptical about remote working, and while I think it is worse in most ways than office working, it's not a lot worse, and the lack of commute is sooo big a benefit that overall it's totally viable.
if your private methods are complicated enough that you're not able to achieve sufficient coverage through public interfaces it's time to refactor. make the private code a library that you can test in isolation.
When I did C++ programming, I would test via public interfaces. If it didn't seem sufficient, or got messy, the reason would always be "This is a big class that is doing a lot of stuff." I would then identify all the things the class was doing, make classes for each one, and have instances of them in the original class as private members.
This way I could test the individual classes easily, and still test the (formerly) big class using only the public interface (with appropriate mocking as needed).
The main thing I learned from unit testing is to pay little attention to the word unit. Insisting on testing the private methods after the exercise above is usually a symptom of aiming for a level of intellectual purity that does not benefit the code.
(Of course, a better solution may be not to use OO to begin with, but that's a whole other discussion).
I worked in a place that disallowed friend functions and classes generally, but explicitly for unit testing. I never wrote another private function. Instead, there was an explosion at the tiny-free-function-that-could-have-been-a-private-function factory, and the parts ended up everywhere.
IDEs are a deep topic but I don't hold such absolute.
For some languages using commercial IDEs is a very smart choice. Refactoring TypeScipt, for example, with Jetbrains IDE is a breeze, and I think it should be available in developers toolkit whatever their preference is.
For some, more dynamic/niche languages (Hello Elixir!), IDEs stand in a way because they take longer to set up and they still don’t produce results as good as glued together scripts and editor macros (side note: macros aren’t only for inserting text, one can pick text under cursor and search for a specific pattern using rg or even send refresh signal to a browser on the other screen).
There are also two other layers that I always mention as arguments against IDEs.
First is that IDEs change often and no matter how much you try your workflows are going to change. It’s hard to get high proficiency when things are changing and having new IDE feature replacing Your Way is a pain and a learning deterrent (thing that I experienced multiple times).
Second is something I call GPS Development. When I work with IDE I tend to not pay attention where am I and where I should be going because hopping navigation is super easy. And then when I am deprived of those tools I’m completely lost and not productive at all. With arguably dumber tools I can open shell and still navigate and edit with whatever editor I have and it still works well. Thing is that with my line of work stuff breaks often and IDEs rightfully decline to work on broken codebase.
My current stance is to use whatever you’re most comfortable with (on a unit by unit basis). Struggling with editor or IDE, even the coolest/smartest one, is going to interrupt your thinking process and cause performance hit much bigger than whatever gains it could ever produce.
In my experience it comes down a lot to what people in a project use. If almost everyone has a properly configured IDE then the code will become almost impossible to navigate without one. If almost everyone uses emacs or vim then there is little risk that the code ends up like that. If you have an IDE it does not matter much how the code is structured as nothing is more than a single click away anyway and you barely notice when clicking if you jumped to somewhere else in the same big file or if you ended up in another file, sometimes in a far away subdirectory. Code structure becomes more obvious and important in my experience when you use a text-editor. For better or worse.
> Thing is that with my line of work stuff breaks often and IDEs rightfully decline to work on broken codebase.
So you throw away smarter tools like IDEs because they do not understand your code when it's broken, but then you start using dumber tools that NEVER understand your code?
I would say an IDE can still do anything your dumber tools can (grep, dumb find-replace) even when your code is broken, but when it's fully working it gives you so many superpowers it's incredible you would not want that.
The thing is that those dumber tools can do smarter things than IDEs.
I had 2 use cases lately - one was to investigate usage of all function bodies that used specific library for the last 6 months. JetBrains IDE has semantic search but it requires a lot of fiddling (syntax feels very foreign, might be Groovy thing or their own DSL and doesn’t span across multiple commits.
Second was to navigate through a custom pre-transpiration layer, which JetBrains could pick up around 30% of the times.
Those are two cases solved quickly with list refiltering (first one took 15 minutes, second took less than 3). And they were portable, as sharing was a simple copy & paste.
The gist of it is that dumb and smart is marketing thing. ripgrep is dumb tool just as fzf is, but they can be made into a very smart setup.
IDEs can make high general work impact but from my experience suck badly when you enter road less traveled.
You don't give enough information to know what you actually did, but I am pretty sure you're splitting hairs here... IntelliJ's "grep" is incredibly fast and has the huge benefit that you can keep an iteractive list of matches and go through them in the search window without leaving the search, even edit the matches manually if you want.
> suck badly when you enter road less traveled.
It seems you just decided that's the case without any sort of rationale behind it. Please give a real example I can try instead of hand waving.
> Using an IDE [is great, in particular for navigating the codebase]
As a longtime and ongoing Emacs user... I have to agree.
Sure, Emacs Can Do It [tm?], but having to setup the appropriate tag system, and ensuring it's kept up-to-date with the codebase is a pain.
For example, our codebase is embedded C with #ifdefs depending on which target we're building from. This means that a naive, cscope(regex)-like search will get confused about which is the proper definition for the active target... when it even can find a definition (it's shocking how often it fails for no clear reason).
So I turned to RTags, which meant having to generate compile-commands.json, which... I'll stop here. Suffice to say setup wasn't trivial, and for some reason it eventually broke and I never bothered again.
LSPs of today have a really large surface area to be implemented as LSPs grew a lot on what they can do. I don't think it's easy to wire down a LSP to any IDE, and lots of small IDEs are written around some text component - like Scintilla. If the text component doesn't have a standard way to be wired up to a LSP, that is already properly maintained, it's really hard to implement it.
emacs 29 comes with eglot built-in. That's the LSP client for emacs and as long as your language has an LSP server, it can hook to it. OP is using C and I am not sure there's a LSP server for C that can work well in emacs, but using Go and Dart and Java seems to work just fine in emacs, just like other IDEs.
Even relatively small, immature, niche CLI text editors start with LSP + tree-sitter support, so I'm not convinced that it would be too much for a mature OS like emacs.
Interesting, I have the opposite experience. Ctags are resilient in face of missing headers, source code preprocessing, syntax errors etc. VSCode usually just gives up. At one point I even had to install the Ctags plugin for VSCode...
Yep. And in case of a big project LSP requires at least all dependencies properly installed right on the machine where you write code. Upgraded OS and something is incompatible for a while? You get no hints now, hope you're not in rush because you'll be running full build to discover your typos. And especially good luck should you want to use LSP on two different projects that require different versions of the same dependency...
I keep on trying to use IDEs and have gone through phases with one IDE or another, but in the end I just keep coming back to Emacs with a GNU Screen in another window.
Debugging by printf is useful in a bunch of places. Debugging via debugger is useful in a bunch of places. Why would I limit myself to only one? (Doesn't matter which one; why would I limit myself?)
That looks like basic encapsulation to me - a consumer of the class can read but not modify the value of chanceOfRain. It also allows the class to change how 'chance of rain' is calculated without changing the public interface of the class.
Pre-emptive encapsulation is a cargo-cult plague/mind-virus. Until someone actually inadvertently fiddles with chanceOfRain (ie actually fucks something up) you are just building a bureaucracy for yourself and your users. And, potentially, you are actively impeding someone who is smarter than you or just disagrees with you on how 'chance of rain' is calculated.
I deal with this kind of bs every day in a large oss C++ codebase - some style guide at Google says everything needs to by default go into an anonymous namespace and so you can't reuse anything that the original braingenius deemed useful. Of course this means very few things are actually decomposed/composable. I have lately resorted to forking and carrying patches that expose all the functionality in the anon namespaces.
I've worked on enough codebases that were made extremely fragile by "smart" people fiddling with (or worse, just relying on) notionally private variables, so I think encapsulation is pretty great. OTOH, I've worked on some fairly large scale js/ts where encapsulation is pretty loose, and 95% of the time it's not an issue.
In general, I find encapsulation to be a poor pattern and a symptom of OOP. OOP encourages thinking of concepts in terms of their thinginess, in which case it can make sense in that paradigm to do something like expose supposedly private data on an object through public methods. The need to conceptualize things in this abstract way is totally artificial, however, because it's often uncalled for and serves to just covering up the actual implementation details. Underneath the abstractions, we have data, and unlike in the physical world, that data is not fuzzy and full of mystery; there's a fixed number of flip-flop circuits in the memory hardware, and each of them has only two states. According to physical memory hardware, there are no "objects" or "classes". There are some tasks that are made easier to grok by designing them with object-orientation in mind, but most of the time thinking of things in terms of shapes of data is actually simpler.
The reason the "chanceOfRain" example is so revealing if this is that there's no reason to treat data like that as if it's an aspect of some entity with its own intimate knowledge. Why have this theoretical class wrapping around weather info as if we must ask a meteorologist? What's wrong with chanceOfRain simply being a function with weather data passed to it, or even just a piece of data that's already been computed by something? The latter options undoubtedly would result in less code and an identical outcome of data at-rest.
What's seemingly wrong with it is this idea you are referring to, which is that the authors of code must know better than the end-developers using said code. This mostly erroneous way of thinking is often justified as a means of preventing other developers from wasting their time doing something "incorrect", but the more distal reason is actually to prevent the code authors from having their time wasted answering questions by other developers encountering bugs doing things those authors don't approve of. Just like DRM, it only serves the authors and punishes fair use, while the "pirate" (in this case the pissed-off developer) finds a way to circumvent the system. If a developer is determined enough, they can write patches to get the behavior they want from the code the author attempted to lock away; in many cases, this is the most appropriate engineering decision since submitting a pull request is usually futile and inviting one's self into a debate with some rando.
There's of course other problems with private members on classes. In many languages, private members on a superclass are not available to the subclass extending it.
The worst part of all of this is how common it is for professional developers to subject their colleagues to this poppycock. It's one thing to try and prevent third-parties from doing "the wrong thing", but it baffles me how developers believe in making access to the "intimate" data highly difficult for other developers working on the same codebase. What the purple f***k? That data wouldn't be "intimate" if developers didn't blindly follow the theology of OOP; the only reason any data can be "intimate" to an object is if that object is thought of as a thing with behavior rather than as a shape of data. Someone may of course argue that encapsulation prevents entanglement, but it actually doesn't; entanglement is based on behavior, not data. At the end of the day, data is data, and your program is millions of bits in memory or storage, none of which have a necessary metaphysical relationship with one another. Behavior, on the other hand, is dependent on other behavior. You can hide away all your instance-specific data, but that will not* stop you from writing code that is highly interdependent. In other words, it doesn't make that much of a difference where you get "chanceOfRain" from, because entanglement doesn't stem from using the wrong data but by making things highly interdependent and not separating concerns. In other words, the argument that private members prevent entanglement is crap. At best, it discourages causing unexpected behavior and limits unexpected breakage, but even that doesn't always work given how common it is for breaking changes to be made to minor or even fix version changes.
In my opinion, data should rarely if ever be made private, and the only thing that might make sense to ever be made private is functions. Note that this suggests the exact opposite of "chanceOfRain" example. Obviously, not all behavior necessarily needs to be exported from a module, but just what is expected to be relevant to the end-developer. The beauty behind this approach is that it becomes a lot easier for an end-developer to patch the code or even just copy the code for functions they don't have explicit access to. Even better if the functions are pure* functions that don't rely on the state of module-level or global data structures.
So many problems would be solved in general if developers would knock it off with OOP and especially privatization. These are legitimate tools to have in programming, but their typical usage is a result of mass psychosis.
I echo the comments on Emacs. Jetbrains tooling is just so /nice/.
You can get your Emacs to act like Jetbrains in a number of ways, but that can sometimes end up being quite complicated. I very much like the experience of opening any Jetbrains IDE and having this working relatively easily compared to Emacs.
All that being said, investing in Emacs gives you benefits you won't find anywhere else.
For example, I was able to make my Emacs find by reference and find by definition functionality work well and exactly the way I wanted it to.
I basically set it up to fall back on resources if the previous one didn't return anything. It went from Language Server Protocol -> CTags -> Regex with rg.
It was also cool to pick multiple sources and priority order them for suggestions. Mine, from highest to lowest priority, were: Language Server Protocol -> local buffer matches -> open buffer matches (or something like that, it's been a while since I touched my config).
This made it so that I'd get suggestions from my LSP for code, but if I were typing a comment and repeating a word I've used before then LSP would come up empty but Emacs would be auto-completing that word for me.
It's weird to see Emacs lumped in with VI in the editor wars rather than being thought of as an IDE. Emacs is the ultimate and original integrated development environment. The joke is that Emacs users have integrated everything into Emacs, so much so that Emacs is the OS.
That's one of the main reasons I picked up Emacs back in the day -- it provided a nice integration with gdb.
Funny enough, Neovim har great LSP and gdb support.
The distinction between an IDE and a text editor now mainly comes down to the configuration time, where both (neo)vim and Emacs support the classical IDE features as long as you configure them to.
The whole point is ignoring the dogmatism of our profession full of various shamans with magic formulas to solve everything from scrum to xp to tdd to rust to functional programming to ood etc.
Then it’s always the same shit, but it’s your fault because you didnt do this or that
I’m always surprised this is controversial but: everything you write in a commit message should be evident from your code and its comments. Nobody should ever have to look at git history to understand a codebase.
It's controversial because it is false. Looking at the code gives you what is happening. The why is almost impossible to put inside. The developers have been trying to cram more and more context into the codebase since the first assembly comments were written. And still we fail.
That's just bad code, and yes, that happens. But if you can write it in a commit message you can capture it somewhere else that people will actually see without having to root around in a completely separate system.
In my experience the implementation is always hairy and because of the immensely complex stacks we are using in modern dev it is full of all kind of weird workarounds. And we don't really have good place to put - I wrote it in this insane and stupid way because some other module that I have no control over crashed except the commit message.
And anyway any project that uses issues tracking also has a lot of context in the issue tracking - by your point of view the developer writing the piece of code that needs commenting should somehow cram the whole issue inside the codebase. We are already using external system anyway - there is no harm in putting information there - the system won't be complete without it anyway.
In my experience (and I have been on any kind of failed at any scale) there is no good way to manage information and context - just slightly less terrible ones.
But if you have a piece of insane code, especially if it's fairly localised, why is it superior to explain it in a commit message and not a comment right there next to it? Save people the head scratching.
And yes - I do exactly think that the developer should cram the whole issue in the codebase! Those are your acceptance tests, or explicit representations of use-cases or strategies, or service layers, or at the very least some comments lamenting what would otherwise be inscrutable. If your codebase doesn't have a first-class way of describing what it does and why, I don't think it's well-factored.
I totally agree that it's easy to mess up, I just continue to think this terrible workflow of externalising key information about code and why it exists is fundamentally defeatist. I don't think we should settle for it.
That is a limitation of the tooling we have unfortunately. Every line in the text files that is not code increases the cognitive load of the person when not directly relevant. And unfortunately a lot of development is done on laptops or otherwise screen restrained machines.
I agree in principle that the metadata and code should be linked and easily pop in view on demand. But that is not how the tooling currently works and we find only crutches.
This is cool paradigm that could make amazing startup. Add in some LLM for buzzword to hook investors. And actually LLM that can trawl all the current sources of knowledge and spitting a one paragraph to glance what and why is going on could even work.
But we can't do it while constraining to text files. We need the equivalent of hypertext for code ... and it will be mess in the end.
With lsp, the gap between IDEs vs text editors is narrowing. While I still prefer Emacs, I’m pragmatic enough to jump on to whatever tool does a better job for a specific task. At times, that is Xcode.
Was also sceptical about ChatGPT and changed my mind like OP. I was less pragmatic on this one and brought ChatGPT over to Emacs https://github.com/xenodium/chatgpt-shell. Pretty happy with the result so far.
The flip side of this, the only times I've run into projects without external build tools, protective githooks or even .editorconfig, it was set up by an JetBrains family dev using the IDE as part of the build process.
Having to start a heavy, proprietary IDE to build code and other assets is a royal pain in the hole.
Nothing makes me feel crazier than trying to get fellow engineers to comment their code. Code alone can only tell you the "what." If the "why" is not obvious, comment it.
I am always dismayed by engineers who vote a hard and unyielding "no" on this issue.I guess ideally your private methods should not be tested directly. If you work at an ideal shop doing ideal things under ideal conditions, please let me know if you are hiring.
In all or most of these cases the tests for private code will hopefully be somewhat temporary; perhaps think of them as scaffolding used during construction or renovation.
- For example, perhaps you have a good reason to write the private methods first and you would like to make sure they are sound before proceeding.
- Perhaps you have a division of labor due to a time crunch and you are writing the private methods while somebody simultaneously writes the public methods.
- Perhaps you are encountering some thorny preexisting code with no test coverage and you would like to just make sure things work.
- Perhaps the public methods are undergoing a lot of flux and you would like to make sure the private methods do not suffer regressions during this flux
- Sometimes it's just easier to test private methods directly rather than indirectly via a public interface. Maybe this is a code smell, but also maybe you don't have time for a full refactor.