Yes, this failure mode is not literally impossible. It also isn't literally impossible that Signal accidentally logs all communications to a central server. And as I said: If you want or need to be that careful, just use a different client.