Yep, humans too have to think before answering most non-trivial questions, and especially the ones that include calculations. So it seems "obvious" that we should try to to give LLMs too some time to think before answering, for example with the popular methods of asking for step-by-step thinking, thinking out loud, and only giving the final answer at the end, and also asking it to proofread and correct it's answers at the end all can help with that.

Pause tokens (thinking tokens) are also an interesting method to achieve that and seems to have a positive effect on performance:

https://arxiv.org/abs/2310.02226