Hacker News new | past | comments | ask | show | jobs | submit login
Crimes with Go Generics (christine.website)
172 points by psxuaw on April 25, 2022 | hide | past | favorite | 61 comments



If the author is reading, I believe the problem noted at the end is actually that the code isn’t going through the thunk API and thus no memoization occurs. (Consider that there is no call to Set in the evaluation tree!)

  fib = func(n int) int {
    if cache[n].o.IsSome() {
      return *cache[n].o.val
    }
    return fib(n-1) + fib(n-2)
  }
Should be:

  fib = func(n int) int {
    return cache[n-1].Force() + cache[n-2].Force()
  }


Oh dear. I pushed an addendum to the article: https://github.com/Xe/site/commit/05135edcbe5e474131c15c2476...

Thanks for pointing that out!


Indeed, that works, and finishes in 0.00 time - here is a Go Playground to prove it (using *T instead of Optional[T]).

https://go.dev/play/p/RxMjU34pjJQ


Unfortunately Go's playground messes with the time, so it's always going to say 0.00s even for the slow version.

Here is a hacky workaround to measure the actual time using a syscall: https://go.dev/play/p/kF6Bh048siE

It shows 979ms for the original version and 0.212ms for the fast version.


It does play with time, but essentially only wall-clock time which isn't relevant here. You can easily make tests show non-zero durations.


Nope.

If you call `time.Sleep()` the reported duration will reflect that call, but it's only simulating the delay. The duration of a CPU-heavy task is always reported as 0.00s.

Demo: https://go.dev/play/p/UZToqFseAwO

Output:

    === RUN   TestSleep // calls time.Sleep(2 * time.Second)
    Real duration: 554µs
    --- PASS: TestSleep (2.00s)
    === RUN   TestBusyLoop // for i := 0; i < 1000000000; i++ { ... }
    Real duration: 390.826ms
    --- PASS: TestBusyLoop (0.00s)
    PASS
The simulation is quite good: if you have multiple threads that use `time.Sleep()`, their output will be interleaved correctly. The output panel will even display the lines one at a time as if it's streaming them from the server, but in reality the entire output is received at once and the "streaming" effect is simulated on the client.

Concurrency is one of Go's core features, so it figures the playground would allow some form of concurrency even when the clock is fake.


Interesting. I suppose that makes sense, to let them speed up execution... I am a bit curious how it'd affect concurrent behavior based on time changes (e.g. if it's cpu-bound for longer than the sleep, does the wake still occur in the middle? Or does it wait to advance time until everything's blocked? What if you runtime.Yield? etc?), but that's certainly an efficient option for a public execution environment like this.


Crimes with Go Generics really is the perfect title for this article.

I chuckled at the juxtaposition of the “expressivity and clarity” comment:

>> you can create composite types (types out of types). This lets you get a lot of expressivity and clarity about how you use Go

Followed by long winded code:

>> However, we can make this a lot more complicated with the power of the Thunk[T] type

And confusion:

>> Why is this so much slower?

This article touches on a lot of the frustration i see with abuse of static typing.

You’re going to write more code and take longer to write it and longer to change it (read: you’re going to cost me more money to build and maintain this code), for the unproven claim that you’ll write less bugs.

The author then goes on to write a logic bug in their memoization code.

But only after they’ve created a generic MPMC queue type by taking an already generic MPMC queue type (a go channel) then writing logic that breaks it (can’t handle an empty queue state) which they code their way out of by throwing more code i have to own.


I'm not sure if you're talking about static typing in Go specifically, but imho static typing and generics make code easier to change.

I dread refactoring code that hasn't made extensive use of types, because it makes it harder to know if you've missed anything conceptually (typing reduces the potential behaviors of a piece of code) or literally (e.g. forgot to update a class).


We’re replying to an article where the author also claimed that but then encountered a bug only in their generic version.


About "Why is this so much slower?" this article is a must-read:

https://planetscale.com/blog/generics-can-make-your-go-code-...


Maybe I am missing something here but as someone who is not an expert in go generics, I don't see any obvious "crimes" with the Option and Queue implementations.

I also could not find where the author explains their "crimes".

What I am missing here?


Not a Go person either but I think the joke with the queue is that Go channels are already exactly that - MPMC queue, and with "generics" baked into the language.

The Option is just a good idea and sum types should have been in the language from the beginning to handle errors instead of product types. But that ship has sailed and sunk.


An Option type without pattern matching is just as useless. Before unwrapping the value, you should check if its None and if you fail to do that, you get a panic. That's literally how pointers already work.


Java doesn't have pattern matching (not until the most recent releases, to be pedantic) and the Optional type has absolutely made my programming safer. You don't need pattern matching if you can use functions like `map` and `orElse` or `orElseThrow`, or just return the optional. I get 99% of what I do in OCaml with 'a Option with just these three functions.


Won't those methods still throw an exception if the object is null though? (Honest question, I have not done much Java since Optional was added and was under the impression that it didn't support implementing methods that wouldn't throw exceptions on null like Go can do with nil)


No, and I think you're misunderstanding how Java optionals work. They're just like any ML-type language's option type - a box which can either have something or nothing.

You choose if you want an exception. Map will only run if the optional contains a non-null value, so it won't throw. orElse will safely give you a value if the optional contains a null, so it won't throw either. The only time you throw is when you use orElseThrow, but that's in the name and you know what you're doing.

Ex.

    Optional<Integer> x = Optional.ofNullable(null);
    // This won't throw!
    x.map(i -> i + 1);
    // This won't throw either, and safeValue will *definitely* be an int!
    int safeValue = x.orElse(5);
    // This *will* throw, but you specify what to throw
    x.orElseThrow(() -> new RuntimeException())
These three methods cover nearly everything I did with options in OCaml as well, so I think that's about everything you need.


Right, but what about `Optional<Integer> x = null`? Is there any static check to make sure this can't happen, or is it something you have to just avoid? I can imagine someone might accidentally return `null` instead of `Optional.ofNullable(null)` in a method that returns an Optional, but maybe in practice catching this with a lint is enough.


> Is there any static check to make sure this can't happen, or is it something you have to just avoid? I can imagine someone might accidentally return `null` instead of `Optional.ofNullable(null)

The latter. Java's Optionals are half-baked and don't provide as much safety as you'd think because it's easy for a `null` value to slip in. Notice how they used `Optional.ofNullable` — a common footgun is using `Optional.of(value)`, which throws a NullPointerException if `value` is null[1].

[1] https://docs.oracle.com/javase/8/docs/api/java/util/Optional...


This is true, but I think half-baked safety is way better than no safety, and in practice I've found that it's actually pretty difficult for a null to slip in, because that only realistically happens when you're directly assigning values. Library code that returns optionals has never caused any issues IME, and custom code that returns optionals is super easy to review because there's only one good way to make an optional - ofNullable.


It’s just something you have to avoid. It’s not hard to avoid though, unless you are doing bizarre and non idiomatic things with Optional. I have never actually seen a NullPointerException resulting from a null Optional, in my code or my coworkers’, and I’ve been writing Java since their inception.


All the Java project I've been on, the convention is to always instanciate containers.


> That's literally how pointers already work.

With the huge difference that Go’s pointers don’t statically require checking if they’re `null`.

Furthermore, you can add functor and monadic APIs to `Option` which provide completely safe (and somewhat efficient) usage patterns even if you build the option out of product types.

Though that isn’t the case here, an other interesting item is that you can build an option type out of non-pointer, thus avoiding the indirection and allocation (though hopefully unlike the C++ committee you don’t do it just so you have a pointer without an allocation)


There is no meaningful null check enforcement like with pattern matching, if you try to access the Option value and fail to handle the error correctly, you either get a panic (from the following null deref) or in this blogpost the zero initialized value (which is honestly worse than an instant crash).

>Furthermore, you can add functor and monadic APIs to `Option`

There's nothing preventing you from defining these functions directly on pointers, they'd be just as safe.


Not the way it’s constructed, but there are safe patterns e.g. providing a callback which is only called when a value is present.

Whether or not that’s the way you want to program is a whole different question, but you can get a level of safety from constructs like this you can’t get from a pointer.


> [...] if you try to access the Option value and fail to handle the error correctly, [...]

It's consistent with the way Go's error handling works. Turns out it's not anywhere near as much of an issue as you might think. In practice you always notice that the function you're about to call returns an error and handle it.

> There's nothing preventing you from defining these functions directly on pointers, they'd be just as safe.

Pointers aren't Optionals. They happen to be nilable, yes, but their semantics are broader. A pointer can be used to avoid copying large structures around. How would you distinguish such a pointer vs one that's used as an Optional?


Not the way it’s constructed, but there are safe patterns e.g. providing a callback which is only called when a value is present.

Whetger or not that’s the way you want to program is a whole different question, but you can get a level of safety from constructs like this you can’t get from a pointer.


The slight advantage here is that the Option returns an error if the value is a zero value (which nil is). Existing linters will throw warnings about not checking errors. I don't believe I've seen a "not-nil" checker, although I could be wrong.


Actually, go generics can do sum types, it’s just not well known or advertised.


No, they can't. Or more precisely, only generic type parameters can be sum types, not actual runtime variables/parameters/return types/struct fields.


> not actual runtime variables/parameters/return types/struct fields.

Is it possible to church-encode these with Go generics?


Yes, probably, but I'm not sure that would be very ergonomic (I don't think you'd want to use it for something like Optional), if I understand correctly how it would work.


I was similarly confused. The patterns look very similar to those used in some other languages (at least superficially), so would have been useful to know why those were considered bad.

As a casual Go user, it wasn't obvious whether those particular implementations were bad, whether the patterns themselves are a bad fit for the Go language, or whether there was just a better (more Go-like) way of doing things.


I think the Option example (despite being the one the author thinks may actually be useful) is the worst one, since it really only replaces existing pointer syntax with named functions, but offers nothing else (well, maybe the Take() call?)

A better function to add to Optional[T] / *T would have been Map:

  func Map[T any, V any] (o *T, foo func (T) V) *V {
    if o == nil {
      return nil
    }
    v := foo(*o)
    return &v
  }
or, with Optional:

  func Map[T any, V any] (o Optional[T], foo func (T) V) Optional[V] {
    if o.IsNone() {
      return Optional[V]{}
    }
    return Optional[V]{foo(o.Yank())}
  }


I think author has gotten out of hand with the avatars. They now have three talking to each other. One was fine, but at this point it's just a distraction. Just write the article.


Author here. Thank you for your feedback. I am working on a fourth character. Your input will be added to my notes so I can make things better for everyone.


I think they should do a podcast together and have Cool Bear on as a guest.


How long before someone figures out how to encode lightweight higher kinded types[1] in Golang. There shall be weeping and gnashing of teeth.

[1] https://ybogomolov.me/01-higher-kinded-types/


> Why is this so much slower?

Just a side note: Go test supports benchmarking (https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-i...) which gives you few useful tools such as `b.ReportAllocs()` and `go test -bench . -cpuprofile cpu.out -memprofile men.out`. These tools can give you far better information than just the execution time of a test case.


Also this article is a must-read to understand about the shortcomings of Go 1.18 implementation of generics:

https://planetscale.com/blog/generics-can-make-your-go-code-...


    Go 1.18 added generics to the language. This allows you to have your types take types as parameters so that you can create composite types (types out of types). This lets you get a lot of expressivity and clarity about how you use Go.
Aren't structs already composite types?


I assume the difference is that without generics you can't freely re-compose a struct type again with different, other types. You have to make a whole new struct type from scratch


The core difference is that structs are static. Once a struct is defined the type are fixed. "composite types" here refers to type that can be dynamically changed when you annotate them. i.e. if you have an custom array type, you can compose them on the fly with MyArray[int] or MyArray[MyCat], structs can't do that.


I guess the author meant "types of types".


She? They?

the website is called christine.website so it is probably not a man


I'd prefer http://pronoun.is/xe/xer/xer/xers/xerself but they/them works too


i see! my apologies, i did look, and I made it as far as your twitter and still didn't parse that.

Please forgive my ignorance! :)


[flagged]


[flagged]


I don't normally care about the author's gender, but I'd argue assuming that is a man is a honest mistake, as long as women in the tech industry is still a minority. Do you really need to go that far? I think that shows how fragile you're.


when i make a mistake, i apologize and correct it.

Especially one as minor as misgendering, which inevitably happens and is easily fixed with a quick: "oh, they*! My apologies!"

so your assertion "i don't care" comes off as flippant and aggressive.


Iain M. Banks once wrote this: "It's brains, not gonads that matters".


The fibonacci should use an array/slice for storage instead of a map(hashmap).


I don’t think the Option presented here is sufficient (it is indeed a “crime”) as you cannot express Some(nil).

But you could probably achieve that using go's not-well-known sum types, e.g. like this

https://github.com/FSMaxB/type-safe-builder-experiment/blob/...


Well, that doesn't really work, as "StaticOptional[T]" can't be used as the type of a variable. It can only be used to create another generic function.

For example, this doesn't work [0]:

  //compilation error: interface contains type constraints
  func TryParse123(s string) StaticOptional[int] {
     if s == "123" {
       return StaticSome[int]{123}
     }
     return StaticNone{}
  }
[0] https://go.dev/play/p/1CnieCLqESC


My bad. I was not aware on incredibly limited go generics are. I tried to make it work despite all restrictions but gave up here https://go.dev/play/p/XVTuJ--ZLgS


I'm not well versed in golang generics (at $WORK we use go, but not generics yet) but if you want type switches you'll need interface{} to do that https://go.dev/play/p/1Tn2O_7vCTQ

Which of course defeats the point of using generics

_edit: clarify that my attempt is a bad practice_


I’m pretty sure that’s an interface union, not a sum type. That said, you could definitely do an option type using interfaces, but it would be less efficient than pointers directly because of the itab overhead.


> I’m pretty sure that’s an interface union, not a sum type.

It's a kind of sum type, but it can only be used to constrain types for a generic parameter.

That is, you can define `func foo[T StaticOptional[int]](x T)` and then call it as `foo(StaticNone{})` or `foo(StaticSome[int]{123})`, but you can't do `func foo() StaticOptional[int]` or even `func foo[T StaticOptional[int]]() T {return StaticNone{} }`.


I think that still eats the itab penalty so as long as the structures wind up with the same gcshape? But now I wonder.


That Option type is using pointer, not foremetioned Zero, &nil and nil are different things.

https://go.dev/play/p/88We4TDhGQc


You can store nil in an Option[any] or in any pointer type such as Option[*int].


The design isn't very reader-friendly to be honest. I really don't like the trend of terminal-inspired designs. Yeah, looks nice and nerdy, but isn't really useful for reading. Monospaced fonts aren't very good for running text as well.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: