I've been comparing it to a very excitable intern. You ask them to explain complicated topic, or design a system, and then they go off and spend three weeks reading blog posts about the subject. When they come back, they eagerly and confidently recite their understanding. Sometimes the information is right, sometimes it's wrong, but they believe themselves to be a newly-minted expert on the subject, so they speak confidently either way. The things they're saying will almost always sound plausible, unless you have a good level of knowledge about it.

If I wouldn't trust an eager intern to educate me on it, or accomplish the task without close supervision, I don't think it's a productive use of an LLM.