Taking a further step back from LLM’s, this is called portfolio / ensemble techniques in the literature.

A common practice in more formal domains is to have a portfolio of solvers and race them, allowing for the first (provably correct) solver to “win”

In less formal domains, adding/removing nodes/trees in an online manner is part of the deployment process for random forests.