We are using Google's related Cloud Run product to do this and have found that, like Cloud Functions, these serverless offerings are a fantastic way to evaluate models in production.
We prefer Cloud Run because it allows us to use docker containers rather than language-specific APIs. We are already using containers for developing and training these models, so it fits nicely in to our workflow.
We prefer Cloud Run because it allows us to use docker containers rather than language-specific APIs. We are already using containers for developing and training these models, so it fits nicely in to our workflow.