When I read your post, I thought 'I bet this guy programs in Haskell'. And seeing your submissions I think I was right!
Humor aside, I agree. I think it's also a big problem in (general) AI: the usual cycle is someone thinks about a clever-sounding but vague and completely arbitrary theory of thought, then starts coding it up and fails to get anything really interesting. Early lisp programs were just one of these attempts at hacking away an AI. I think what we need is an abstract, mathematical language to describe intelligence. AIXI, as mathematical as it may be, doesn't fit the bill; it's one particular AI, not a language sufficient to describe AIs and their properties.
Humor aside, I agree. I think it's also a big problem in (general) AI: the usual cycle is someone thinks about a clever-sounding but vague and completely arbitrary theory of thought, then starts coding it up and fails to get anything really interesting. Early lisp programs were just one of these attempts at hacking away an AI. I think what we need is an abstract, mathematical language to describe intelligence. AIXI, as mathematical as it may be, doesn't fit the bill; it's one particular AI, not a language sufficient to describe AIs and their properties.