Sure, perhaps my example was too extreme. What about:
$ gitgpt commit files with msg cleaning repo files and push
git commit -m msg
git clean -fdx
git push
Without `git-add`, 1st one is a NO-OP, 2nd is a destructive action, 3rd is a NO-OP. All vaguely related to the topic at hand, "hallucinating" such a destructive action seems at least plausible.
The GitHub Copilot CLI tool that is in beta, along with other tools, will show you the command and then make you manually choose to run, redo or abort.
In practice I have yet to accidentally perform any destructive operations. This makes intuitive sense because those seem like unlikely completions for a model that has been fine-tuned on “helpful cli recommendations”!