Note that code-davinci-002, despite the confusing name, is the actual GPT-3.5 base model, which only does completions and does not have any mode collapse. And it is still available via Azure, as far as I can tell. Text-davinci-003 is a fine-tuned version of it.

More info:

https://platform.openai.com/docs/model-index-for-researchers