When you combine the output available now with the fact that a surprising number of humans on the internet would probably fail a written Turing test, it's definitely useful at least for denial-of-service style mischief.
It seems to be supervisable at this point, specifically for shorter responses. I was thinking if you're trying to AstroTurf or some other such form of maliciously using message boards/forums, rather than giving free reign you could present an operator with the prompting text and some arbitrary number of generated replies to choose from. Still entails some leg work but will likely yield more believable responses more frequently.
Sure this is only economical if at all when replying to shorter prompts, but that is likely going to be the majority on most forums and message boards.