A simple way to think of a nuclear thermal rocket is that it's a nuclear reactor where the coolant is vented into the vacuum. (At very high temperatures, through a rocket nozzle, therefore it produces thrust.)

So yes, it pretty much by definition has no cooling issues. If you'd stay too hot, you'd just pump more fuel to cool your reactor better, and get more thrust.

(One flipside to this is that you cannot just turn it off. After a long burn, there will be decay heat you will have to dispose of, potentially for weeks. NTRs design around this with complex, slow shutoff sequences, or in the case of disposable stages, by making sure the stage will not be near anything important as it melts down.)