You're correct that it's an exercise in introspection, rather than relying on the AI's own knowledge. It was clear to me when trying to write explanations to its questions where my understanding was starting to get fuzzy and hand wavy.
A nice bonus was to get it produce a scorecard of correct, nearly correct and incorrect explanations. I could see these as a good jumping off point for me to do more learning/research. Though I suspect the AI would be less accurate at this in a more niche topic than I chose (refrigeration).
That is an interesting conversation. I like that you had the chat bot follow up with questions forcing you to dig deeper. The summary at the end is a good idea as well.
I've been asking the chatbot to point out my assumptions and to point out weaknesses or errors in my reasoning. That has been pretty useful in uncovering aspects of my thought that aren't as foundational or coherent/consistent as I expected. It also doesn't require the AI to be an expert in order to give me feedback, it just points out unchallenged assertions and errors in logic.
I think this is a potentially powerful use for AIs. They can patiently and politely nitpick and point out subtle errors in our thinking process, providing unbiased critical feedback. That is almost as valuable as having them be perfect oracles that can answer any possible question.
> It helps that refrigeration has always sort of blown my mind, so this was an interesting topic as well.
Same! It's been in the front of mind because I've recently been watching Hyperspace Pirate's videos on building a DIY cryocooler (and have been struggling to follow it at a technical level). You might enjoy them too: https://www.youtube.com/watch?v=7QZrHzd3RA8
You're correct that it's an exercise in introspection, rather than relying on the AI's own knowledge. It was clear to me when trying to write explanations to its questions where my understanding was starting to get fuzzy and hand wavy.
A nice bonus was to get it produce a scorecard of correct, nearly correct and incorrect explanations. I could see these as a good jumping off point for me to do more learning/research. Though I suspect the AI would be less accurate at this in a more niche topic than I chose (refrigeration).