For the record LLM are theoretically fully deterministic, but non-deterministic in practice. First, some randomness is deliberately set via 'temperature', and second some randomness comes from things like the order of floating point operations when you divide the model on your GPU(s) without being super careful about it.
LLMs output probability distribution of the next token. And that automatically makes them non-deterministic. You can make their output deterministic by greedy sampling, fixing the seed of a pseudorandom generator, or by computing exponentially growing probability distribution of all possible continuations, but it doesn't change the fact that LLMs produce probability distributions you need to sample somehow to get a definite result.
