Nice article, thanks for publishing this. However:
>This panic is caused by dereferencing a nil pointer, as indicated by the first line of the output. These types of errors are much less common in Go than in other languages like C or Java thanks to Go’s idioms around error handling.
I often see assertions made like this about Go. My experience at companies which use Go as their primary development language with reasonably-sized teams (30-50 people working on hundreds/thousands of Go programs) does not agree with this. I would say nil pointer dereference errors are just as common as if they were writing code in C or Java. I don't have hard data to back this up though; perhaps you do.
> If a function could fail, the function must return an error as its last return value. The caller should immediately check for errors from that function.
In practice, in idiomatic code it is quite rare for a function that returns a `(* Something, error)` to return `(nil, nil)`, which are the cases where a nil pointer dereference would most often happen. The other case is not _immediately_ checking errors. I would say both of these cases are not idiomatic Go. (Yes, there are exceptions.)
The other cases where I've seen nil dereferences pop up most often:
(1) the zero value for maps is nil, and a nil map is mostly not usable. `m[50] = "foo"` will panic with a nil dereference if you've not initialized `m`. This is one of the most annoying things about Go.
(2) not checking for presence when pulling out of a map. If you have `map[string]* Something` and do `x := m["nonexistent"]` and try to use `x`, it'll blow up. Always use the `x, ok := m["nonexistent"]` form instead, and deal with stuff when `ok == false`.
(3) nil receivers. `var x *Something`, then `x.Method()` -- the `Method()` method _can_ deal when `x == nil`, but most code does not.
My reasons for citing C and Java:
C doesn't have multiple return values, and kinds of C code deals with errors differently. Lots of stuff just returns `NULL` and all too often things don't check for it. (`malloc()` is a perfect example of this -- it rarely fails but a _lot_ of code doesn't check for errors. Now extrapolate that to everything :)
For Java, it's because the try-catch control structure complicates program flow. If you have a large block of code in a try-catch, it is very easy for a different line of code to throw an exception than you expected, and as a result an object you attempt to dereference later is null and you get an NPE. There are ways to deal with this, of course, but in my experience most people are pretty lazy about the non-happy path.
EDIT: formatting. how can a site about programming have such terrible support for typing in code
My 7 years of C++ and 7 yesterday of C# and 5 years of go agree with you. Many more nil pointer exceptions in the former and almost none in the later in similar size (large) codebases.
Most of my experience is in C#, and NullReferenceException is, by far, the most frequent exception that ever happens. Curious about if your experience with it is different, and if it is, how.
One of the most common errors that i used to encounter when i first started with golang was- accessing a map without initializing it first. For example:
var m map[string]int
m["name"] = 23
which resulted in "panic: assignment to entry in nil map".
Nil pointer access errors are relatively easy to debug when in Go world, as compared to in CGO. Like, we are using zmq via CGO in one of our project and when these errors occured, we had to use gdb to debug (apart from logging). At one time we even used rr.
I think it depends on what you're doing. In my old org, we often avoided pointers. Or rather, we used them when we needed them. Because of this, I'd wager easily 50% of our codebase was not using pointers at all and was completely type safe.
Though regardless, after using Rust for a while, i hope we can find a way to bring that safety to Go. I use a compiler and not JS/Python/etc for a reason, after all. I want compile time safety, all the time.
I really fail to see how "Go's idioms around error handling" would make NPE rarer than in Java either.
In java, an error is an exception and will bubble up to the handler, it has very little reason to "birth" null references.
In Go, an error is a non-nil value next to a zero (often nil) value, and a success is the reverse, so error handling always injects new nils which you're supposed to carefully not used (since no sum types).
My experience is that it's slightly less common in Go than in C or Java, but quite a lot easier to debug than either of the other two. My explanation for why it's less common in Go is that fewer things are pointers, especially compared to Java, and that the things that the cause is more immediately apparent, especially compared to C. Of course, nil/null is still an unnecessary problem in all of these languages, in the sense that there are language features that render these problems obsolete; however, these languages (especially C and Java) aren't likely to adopt these features or at least not to adopt them and completely sunset nil/null for compatibility reasons.
As a not-yet-experienced Go programmer, I have to say it'd make it way more palatable if the reference to the error in my code could be somehow extracted and displayed in a more obvious fashion. I've had panic happen in Go code before (most of the time when trying to read the body of a http response that happened to be nil for whatever reason) and I was kind of lost in a trace of errors happening deeper in the standard library that I mostly didn't care about at first sight (until a few lines down, I could find a reference to my own program).
That being said, Go is still a pretty good language when it comes to tooling and error handling / reporting.
To offer a counterargument, I've always considered Go's panics to be pretty readable because you can read the top line to get the error and then one of the next lines down (I forget which but after a while it tends to stand out) to see the line where the error was raised. The rest of the stack is if you need to trace the error back through the calling routines (I rarely need to do this but it's useful if the panic isn't being raised by your own code).
Granted things get a little more tricky when you start multiplying the number of concurrent goroutines. I've sometimes ended up with several terminal screens full text. But so long as you can find the start of the panic output then you can generally find what you want without too much difficulty.
Have you ever looked at typical template compiler errors in C++? The same problem occurs, I suppose, but then much worse. I regularly have multiple screens of text where your own single line of code is hidden like a needle in a haystack. Clang seems to do this _slightly_ better than gcc, but it's both bad.
On the whole I am very happy with Go, but this did bite me recently. We're using the X509 package which is returning an error on a handful of CA certificates due to characters that are not technically allowed in an ASN.1 PrintableString field (E.G `&`). Sure, fine, we've already forked the package so it's easy to make the change to be more lenient. Except errors don't contain any information about the origin of the error in source code, so it's been down to painful debugging and source code digging to find where this check is being made.
If anyone has a less painful way to do this, please halp.
Part of the reason has to do with Go's original primary use case: a server handling thousands of requests. In that environment a traditional debugger, at least in my experience, isn't super useful, because it models a single-stepped workflow but the program is massively concurrent. (also pausing a production app is probably a bad idea)
The kinds of bugs you run into are different and harder to track down, so Go has invested resources into some of those problems (for example the race detector, and automatic detection of concurrent map misuse) and you tend to rely on logging, profiling, tracing and stats to solve bugs.
I think there are plans to make an official debugger it just hasn't been prioritized.
> I think there are plans to make an official debugger it just hasn't been prioritized.
There was a nascent project inside the Go team a couple of years ago (called ergo I think?) but I believe it was shelved in favor of Delve, which was much further along.
It's probably safe at this point to consider Delve the de facto standard Go debugger.
Panics can be caught and handled by parent callers and thus behave a little bit like exceptions (not as pretty I'll grant you, but still better than the "no way out other than aborting the process" approach it seems at first glance). However you generally you don't want to catch panics except under rare conditions because panics are intended to be used in "the world is ending" kind of scenarios (it doesn't always work out this way but that is the theory).
For general error handling there is an error object that gets returned by functions. It leads to the often moaned about syntax below:
if err != nil {
return err
}
Yeah it's ugly. Yeah other languages have try / etc. But for all of it's sins I've found it still gets the job done.
I don't mind that golang avoids exceptions (it would be better if it didn't have them at all) but I don't understand why they looked around the language landscape and landed on C style return codes as the right path forward.
Monadic error handling has all the supposed benefits the return value style errors but allows for easy composition. After years I'm still infuriated with error handling in golang and it was such an easy thing to avoid.
It's easy to criticise this from a visual glance but you have to remember that proper error handling in any language is hard work and often produces ugly code where you need to account for multiple edge cases - many of which aren't serious enough to cause an exception but would still break the program's logic. All Go does is group these conditions into a standard interface.
Furthermore, because Go does unify these conditions, I've found it effective for handling a whole array of differing edge cases that I might have otherwise overlooked. In that regard I've anecdotally found it more effective than most of the habits you see in other languages (I can't say all languages because I haven't used every language out there, but I have developed in excess of a dozen different programming languages over the last 30 years).
In short, in Go error handling becomes a first class citizen rather than a byproduct you begrudgingly need to check for.
Also while it might not be to everyone's tastes, adding "if err != nil {" really does add practically nothing to your overall dev time in real terms.
But I'm not here to advocate one method of error handling over another, nor to try and convince anyone to use Go who might not have otherwise. Obviously you can see I get a little frustrated when people judge it purely on face value but that's because I have spent some time with the language and found it to be surprisingly effective at delivering a stable product.
> Also while it might not be to everyone's tastes, adding "if err != nil {" really does add practically nothing to your overall dev time in real terms.
But how do you compose functions then. You can't write:
f(g(h(...)))
> when people judge it purely on face value
Yes, I can see that is frustrating. But I already know the style of programming that requires if-statements instead of exceptions, as I have been programming in C a lot, and I don't want to go back to that style because I'm lazy and I'm sure I will forget to handle some errors. I want my language to deal with this. Honestly, is that really too much to ask? Even javascript has exceptions.
> Anyway, I'm not quite sure how is that different from, say, Ruby's begin...rescue.
Ruby lets you write foo(bar(baz())), the rescue block will just coalesce errors in any of the three calls.
And because Go has no generics and its errors are not reified (handled through MRV which is a special language construct) you can't build abstractions like flatmap/then/>>= which would let you compose these functions (or function calls), you have to handroll every instance.
> As a special case, Go does allow foo(bar()) if foo takes two args
Good to know (though somewhat disgusting), but for error-returning functions I would assume the external ones would take a single argument aka in foo(bar(baz()) I'd expect
> I don't want to go back to that style because I'm lazy and I'm sure I will forget to handle some errors.
Because errors are a returned interface you don't forget to handle them. In fact you have to explicitly tell the language not to handle them by using a _ token on the function call.
However it sounds like your argument is more personal preference regarding coding style (eg it seems you tend to favour Sexpression style of syntax over the C-style of syntax) and that's reasonable enough. I do write a lot of code like that too - albeit not in Go.
Sure, and then you have no idea what was executed and what failed. Did you run 9 functions or 1? What do you need to roll back? How do you handle the error?
This is the problem with exceptions. They make it too easy to be lazy with your error handling.... Most of the time it's just catch and log, because the code itself has no way of knowing what failed and what succeeded. This is how you get your program in a bad state.... Because maybe you uploaded the file but didn't set the metadata in the API, because the connection broke between those steps.
With go's error handling it forces you to think about "what happens if the code fails here". Its always obvious what code had been executed and what has not.
To get that behavior in exception oriented languages, you'd have to wrap every call in try/catch, which ends up just as verbose as go, if not worse.
> Sure, and then you have no idea what was executed and what failed. Did you run 9 functions or 1? What do you need to roll back? How do you handle the error?
> […]
> With go's error handling it forces you to think about "what happens if the code fails here".
Did it run 9 functions or just 1? What do you need to roll back? How do you handle the error?
The inner code could have just bubbled an error by hand from a deeper call (or the library could even have used panic/recover internally to do so, the stdlib used to do that). You have no more idea than in the exceptions-based code unless you have the source available right there, which you'd then also have in an exceptions-based language.
> To get that behavior in exception oriented languages, you'd have to wrap every call in try/catch, which ends up just as verbose as go, if not worse.
So your argument in support of Go's error handling is that the very worst case of exceptions-based error handling you can imagine is about as bad as the baseline case of Go's?
"What failed?" is recorded in the exception, along with the stack trace of how we got there. try...finally can perform best-effort cleanup, it's just like "defer" with block scope that fits the rest of the language. But it's not that unusual for server hardware to abruptly die, and no error handler can recover from that. "Uploaded file but didn't set metadata" has to be handled in the design phase. Another server needs to notice there may be some unfinished work (often by retrying idempotently).
In practice, what you end up doing is adding the `errcheck` linter to your build process and it finds all instances of you not dealing with an error return and it's "just-as-good" as a compile time check.
If it is not mature enough for you now - it will never be. Right now it is already used in many huge IT companies and extremly popular in China. Writing that if-construct after every function is the design, not the lack of it.
> However you generally you don't want to catch panics except under rare conditions because panics are intended to be used in "the world is ending" kind of scenarios
Strange, I've always heard the opposite: if you write a function that can panic, you always want to handle it in your code that calls the function (which should be the only code that could) because panics should generally not be exposed across a public interface, including the boundary between the top-level Go code to the outside world.
panics are like asserts. They're here to make sure the program is being used correctly. There is no handling of them, if it's not used correctly you crash.
Of course you do not want to use panic in sensitive code. The best example is looking at how panic() is used in crypto/tls (which has a lot of sensitive code! You really don't want a client to be able to crash a server): https://gist.github.com/mimoo/93c33d88222c87fb384b4ca3b04fb5...
That's simply not true. There is no recovering from a failed assert. Go's recover() allows recovering from panics.
Further, it is considered good practice to recover from panics within library API boundaries, and return errors across the API instead. "The convention in the Go libraries is that even when a package uses panic internally, its external API still presents explicit error return values."[1]
Yes, if you use panics for control flow in your package, you need to recover from them and hide that fact from your API. That is not the same as saying that you should recover from any panics that happen inside your library. Unexpected panics basically always indicate a programmer error.
Can you give me example in the standard library where they recover from a panic? So far I've only seen recovery through error handling. I think the crypto/tls example is pretty telling on the way they're using panic for sensitive code. No recovery there.
The encoding/json package is interesting because it uses both panic() and recover() internally in a way more reminiscent to how exceptions are used in other languages (that is, for control flow).
>This panic is caused by dereferencing a nil pointer, as indicated by the first line of the output. These types of errors are much less common in Go than in other languages like C or Java thanks to Go’s idioms around error handling.
I often see assertions made like this about Go. My experience at companies which use Go as their primary development language with reasonably-sized teams (30-50 people working on hundreds/thousands of Go programs) does not agree with this. I would say nil pointer dereference errors are just as common as if they were writing code in C or Java. I don't have hard data to back this up though; perhaps you do.