But we have a pretty clear idea of what we want. Just looking at the remarkably intelligent and spectacularly unintelligent should give a definite picture to work on. (When the spectacularly unintelligent is responsible for important resources a sense of urgency can easily be added.)
> integrating sensory input with action and reward mechanisms
There you reveal you may not be speaking about what others will call intelligence (just read the paragraph above).
Or, if you meant that "intelligence would just be cybernetics" (as was already a supposition in 1956), the problem remains that we are interested in an "ontology refiner", so the primary question would remain of how to create an ontology refiner from sheer cybernetics. And if it made sense to have a cybernetic implementation spawn it, instead of implementing the ontology refiner and its feedback parts in parallel directly.
> But we have a pretty clear idea of what we want.
The bellwether for me personally is waiting for a system that can generate something conceptually novel. Something like the move from the real to the complex number systems. Or the move from Newtonian motion to relativistic understanding. Maybe systems already have such insights but don't have the vocabulary to explain it.
A system that when presented with a problem we don't even know how to tackle, can "invent" the tools/approach needed to solve the problem.
In terms of the Langland problem in math, a system that can define a new landmass there or a new bridge between existing domains.
But we have a pretty clear idea of what we want. Just looking at the remarkably intelligent and spectacularly unintelligent should give a definite picture to work on. (When the spectacularly unintelligent is responsible for important resources a sense of urgency can easily be added.)
> integrating sensory input with action and reward mechanisms
There you reveal you may not be speaking about what others will call intelligence (just read the paragraph above).
Or, if you meant that "intelligence would just be cybernetics" (as was already a supposition in 1956), the problem remains that we are interested in an "ontology refiner", so the primary question would remain of how to create an ontology refiner from sheer cybernetics. And if it made sense to have a cybernetic implementation spawn it, instead of implementing the ontology refiner and its feedback parts in parallel directly.