There is a fair amount of existing work related to planning in continuous and unbounded spaces under uncertainty. It seems likely some of the existing techniques combined with modern language models could be quite effective. A couple entry points:

Deep Reinforcement Learning with an Unbounded Action Space - https://arxiv.org/abs/1511.04636v3

Efficient Planning in a Compact Latent Action Space - https://arxiv.org/abs/2208.10291