The point of this code release seems to be to release Google's precise logic. That you may incorporate it into something else is, IMHO, less interesting; we've got plenty of other solutions that "do robots.txt" well enough. If it was just about that, Google's release of this would not be worth anything. The point is so that non-Google parties can see exactly what Google is seeing in your robots.txt.
That's why I'm saying there's no point trying to re-implement this. If you were going to re-implement this, there's probably already a library that will work well enough for you. The value here is solely in being exactly what Google uses; anything that is a "re-implementation" of this code but isn't exactly what Google uses is missing the point.
If they formalize it into a spec, others may then implement the spec, but they can and should do that by implementing the spec, not porting this code.
As I understand the point about Go complaint is to parse actual real world robots.txt. For which you don't need to behave exactly as this library does.
That's why I'm saying there's no point trying to re-implement this. If you were going to re-implement this, there's probably already a library that will work well enough for you. The value here is solely in being exactly what Google uses; anything that is a "re-implementation" of this code but isn't exactly what Google uses is missing the point.
If they formalize it into a spec, others may then implement the spec, but they can and should do that by implementing the spec, not porting this code.