This is a niche complaint, but I get frustrated at how imprecise open AI's papers are. When they describe the model architecture, it's never precise enough to reproduce exactly what they did. I mean, it pretty much never is in ML papers[0], but open AI's bigger products are worse than average with it. And it makes sense, since they're trying to be concise and still spend time on all the other important stuff besides methods, but it still frustrates me quite a bit.

[0] Which is why releasing your code is so beneficial.