Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine learning for pattern recognition tasks of any kind.
The currently trendy "AI" looks more like massive data mining with powerful ML to me, that's very good for certain tasks but brings us nowhere near real AI. The knowledge representation problem has not yet been solved. In order to get even just a convincing simulation of AI, let alone real AI, we need a large common sense knowledge base / computational ontology.
One perhaps promising approach are geometric meaning theories/concept representations that allow for logical combinations. For example, Diederik Aerts works in this area. However, to be honest, I don't have the math skills to evaluate his approach. Generally speaking, logical modelling is too limited for good concept representations - especially classical 90s AI like in default reasoning and other nonmonotonic logics -, whereas traditionally geometric representations suffer from problems with representing logical inference and quantification. IMHO, that's a problem worth looking at. (Admittedly, I'm a bit biased towards symbolic AI like the people in this article.)
>> Generally speaking, logical modelling is too limited for good concept representations
In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language. The problem is that in practice it is very hard to transfer into a logic language all the knowledge you might need for a useful system.
In fact, this was one motivation for at least one branch of machine learning- the work from Ryszard Michalski, Claude Sammut, Ranan Banerji, Steven Vere, Brian Cohen and later rule-learners like Quinlan's decision tree learners and of course all the later work in Inductive Logic Programming by Plotkin, Shapiro, Muggleton etc. These are all rule-learning systems, that can avoid the knowledge acquisition bottleneck that probably did in for purely rule-based expert systems in the '90s (i.e. getting experts to transfer their knowledge into rules).
I rather agree that further progress in AI will require a, let's say, synchretistic approach- like I say in another comment, the obvious thing for me is to use deep learning for perception, logic for inference and probabilistic modelling to deal with the noisy world. There are some people working in that sort of direction, with different amounts of emphasis on each of the three approaches. For instance, Josh Tenenbaum at MIT, Evans and Grefenstette at DeepMind, Luc De Raedt at KU Leuven, Kristian Kersting at Dortmund, Lise Getoor also at MIT, Pedro Domingos at Washington, and many others.
Apologies for all the name-dropping without links. Let me know and I can provide them if required.
I agree totally that a "non-pure" deep learning approach is going to be necessary in pushing the field forward. World shattering results from new deep architectures like AlexNet are becoming more and more rare and improvements are becoming incremental. Language models, for example, showed some impressive results in recent years, but we're still a long long way from generating realistic and useful text. I'm not sure if throwing more data at larger architectures will get us there. Maybe, maybe not.
The work by Tenenbaum at MIT is a particularly interesting mix of psychology, computational statistics, and neuroscience. It really challenges the commonly repeated notion that neural networks are completely unrelated to actual brains.
> In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language.
Is there any proof about an existing non-leaky abstraction between natural language and First Order Logic (or logic in general)? Afaik there is none, but I might me wrong. For example (one amongst thousands): “I don’t like you” can actually mean (and it often does) the exact opposite, and the same can be said of a simple statement like “I am ok” which also can mean the exact opposite. Natural language is a very hard problem.
Does it matter, though? Understanding sarcasm or other forms of language twisting seems like only important for a very advanced and specific set of applications. There's another post in the front page about the death of a gorilla that was able to do signs language. I'm sure nobody cared if he could catch jokes.
Well, going by formal language theory, FOL is equivalent to a universal Turing machine, which is probably also the full expressive power of human language. Therefore, everything that can be said in human language should in principle be possible to say in FOL. I don't think this is controversial.
Notice again that I don't think this is feasible in practice. At least not with hand-crafted FOL expressions and certainly not with existing technology.
I'm not sure what "non-leaky abstraction" means. Would formal language theory fit the bill?
Yeah but humans don't have a single consistent logical model for all reasoning - i believe that all my models can be in fol, but i don't believe that they can be unified in fol
Oh, I'm not saying anything about how humans think, or model the world. FOL is a formal language. It can express the same things that natural language can express and you can do reasoning in it (actually- let's say "inference" because reasoning is a bit of a loaded term), but I make no claim that it's how humans really think.
I mean, I don't believe that is true any more that I believe calculus, or linear algebra are the way we think. Those are just formal systems that we can use to get various results about the world- but is that how we really think inside our heads? I kind of doubt it.
I'm just talking about the expressive power of first-order predicate calculus here, not about whether it's a good model for human intelligence.
I'm going to assume you mean the computational complexity of working with such an expressive representation. If so, then that is correct. historically, working such an expressive representation blows out computationally. But, we've resolved that issue with our graph engine. that is, its been shown that algorithms take time proportional to N-cubed where N is the number of graphs in the knowledge base. with our graph engine, we encode graph structures and ontology in a Cognitive Signature, and we can find closely matching signatures in log(N) time. the log-time algorithms scale to the size of the web.
I suspect you may have a hard time representing all the implications, subtexts, and allusions in a play by Shakespeare with First Order Logic.
It seems more likely that First Order Logic can be represented by a subset of natural language.
Otherwise, I agree. I don't understand why everyone has been concentrating so hard on probabilistic ML when there are many applications where inductive inference is more efficient and powerful.
> In principle, First Order Logic and equivalent languages can represent anything that can be represented in natural language.
I doubt this is the case since natural language makes a lot of use of metaphors, and our concepts are probably mostly based on the types of bodies we have and how we interact with the world. See embodied cognition and writings of Lakoff and Mark Johnson.
Look into the field of computational semantics. There is absolutely nothing in your comment which cannot be represented in existing logics. Metaphors and such are no problem.
There is ambiguity in natural languages so the determination of underlying meaning is at best probabilistic, but we have ways of handling that too.
Lexical and computational semantics may be superior to FOL, (for machine translation, at least) but they've been around now for a very long time without showing much promise toward thawing past AI Winters. I've not heard anyone propose their adoption as the missing next step toward modeling general knowledge across all AI problem spaces.
In fact, populating a LS/CS knowledgebase with requisite facts was a game stopper when they were first invented and still pose challenges of scale that remain unsolved decades later. If it accomplished anything, Cyc proved that manual curation alone of a rich database of propositions is depressingly infeasible. So how does one automatically curate a general fact KB, no matter what semantic model the facts take? I don't think expressiveness of the propositional representation will ever decide that question.
I’m not sure what you are saying. Computational semantics parses sentences to possible interpretations in first order, or higher order logics. It is not some sort of alternative to FOL. And it is still a rapidly developing field.
I'm trying to suggest (perhaps badly) that formal models of logic modeling like LS / CS have an innate achilles heel (from what I've seen): either A) the knowledgebase is precise but tiny, and will remain so because it had to be meticulously hand crafted to comply with a formal semantic model, or B) the KB is representationally sloppy but big and scaleable, because it was populated automatically and informally (probabilistically), thereby forever limiting its amenability to precise models of reasoning. To wit, you may have your cake, but the morsel is so tiny you'll starve.
I think this tradeoff in quality vs quantity has been central to the inability of most Good Old Fashioned AI (GOFAI) formal symbolic methods to 1) scale up beyond toy academic clean-room problems (from the admittedly little I've seen of commercial LS/CS), as well as 2) support the flexibility and imprecision needed to deal with a messy ill-defined very big world with a seemingly unknowable number of unknowns.
It may be that this venerable yin v yang of knowledge modeling will forever bedevil AI. I don't have the sense that any techniques in the past 30 years have really made headway in breaking this log jam. Deep nets have just tilted the playing field in its favor, so long as we're willing to ignore fussy details like dependence, FOL, and causality.
I think this is the perspective the "Society of the Mind" book by Marvin Minsky took at their time. When a software like Mathematica makes symbolic operations it is doing AI, the same for things like the Z3 solver.
Very astute. Some folks here at Kyndi used to work with Minsky. One of our favorite quotes is: "what magical trick makes us intelligence? The trick is that there is no trick. The power of intelligence stems form our vast diversity, not from any single, perfect principle." What we see now in AI/ML is not diversity, but rather attempts at the single perfect principle. We don't believe that is the path forward. Rather, you need to figure out how to tightly couple different approaches (e.g., symbolic and statistical learning approaches) to form a more powerful, flexible, and explainable model of AI.
AGI will need to be able to learn and represent any symbol system, just like we can. And it doesn't have to be that good at it(we arn't). There's millions of knowledge bases all around. You'll need something with a generative imagination. We model so much more than labels and relation ships.
Imagine a speech, by president Obama. Where you can actually hear his words. or imagine a ball dropping. We just use words to convey the basic idea. How many hairs does a bald person have?
And if we have a misunderstanding we just adjust our system. New word we use context, including social context etc.
Let me suggest something that will turn everything upside down...
How much of our “logical reasoning” about the world is just really strong correlations?
Look, AlphaGo Zero beat the best chess programs by just precomputing MCTS. What does that say about finding the optimal solution to a problem?
It seems MCTS beats even AlphaBeta search, by far.
So when you talk about applying rules, you’re using a system that tried to reduce the system to a few rules and maybe recursively use them. That may work for some things. Certainly it seems to work amazingly for physics. But that’s the unreasonable effectiveness of mathematics.
Ultimately what if I suggested that “understanding something” with huge vectors or monte carlo search is actually far deeper understanding than a few rules?
I am talking about the actual territory not the map. Yes the model is neat and tidy but the model is just simple enough so that humans can understand!
What if an AI can no more explain its solutions to a human than a human can explain human decisions to a cat?
In general, I agree that for avery complicated task that a machine intelligence performs far better than any human being, human-like reasoning might be just impossible. The logical formula might be too long to be reasonable to any human.
But from another side, current AI has a hard time to perform ordinary human reasoning at all. Those reasoning mostly contains a few logical clauses. Why?
Human like logic usually hide the common sense or context in other words. There is no absolute logic like FOL when we talk about human like logic. The traditional rule based AI system fails because of that.
On the other hand, deep learning are very strong at representing these hidden common sense and context. Look at the word2vec.
We now are at a point to combine these two approaches together not to build AlphaGo, but enable ordinary human like reasoning.
That's like saying that causation doesn't matter; correlation will suffice. But the causal directionality of separating subject from object in a relation is essential to forming or sharing an idea usefully. In the synthetic game worlds you mention, causality is decided by play sequence. But at no point does the agent ever have to employ abstraction, or 'think about thinking', which diminishes the needed depth of thought greatly.
No, probability alone simply isn't expressive enough to support higher cognition. At best it will deliver a purely reactive agent, no more cognizant than Kahnemann's knee-jerk level of thinking.
Funny that you say that. David Hume said that you can’t meaningfully prove causation, only describe things and speak about about correlation.
Also time is a local phenemenon. You can’t simply compare A and B time if A and B are really far apart. When you get to really small scales you would be hard pressed to find which event happened before the other, so you get uncaused events like Virtual Particles in quantum mechanics.
And when you have tons of complexity then what do you mean by cause and effect instead of correlation?
I won't challenge Hume; I'm referring only to the fact that a causal relation implies necessity: B can't happen unless A acted to cause it. Probability can't provide this insight, it can only encourage or discourage it. The agent must formulate a mental model of the relation in order to hypothecate the presence of necessity / causality between a pair of events. A model that lacks that ability (e.g. probability) can lead to all sorts of nonsense in interpreting events, like proposing causality. Or at least meaning by connecting a baseball pitcher's hat color with the weather on game day.
BTW, precision isn't necessary for relative time to have great value in a huge fraction of possible world events. Obviously all you need is relative temporal sequence to disprove causality - since the batter can't hit until the pitcher throws.
In a great many event pairs, a lot of temporal precision Is unnecessary. Often it's only approximate relative time that matters, since the mental task intuits all the temporal constraints needed for understanding event coupling, like knowing the parent child relation implies the relative ages of each and much more latent info that may or not be relevant to a given interpretive task at hand.
And what I am saying is that when you say A implies B you’re making a ton of assumptions. If those assumptions are violated (eg all swans are white, all men are taller than all women) then suddenly you have caveats and hedges.
So what you actually use is not Propositional Logic / Boolean Algebra but Bayes’ Theorem. You only use rules of inference because you make assumptions.
I am saying that may work sometimes but those models are not exact.
> Maybe good AI requires a combination of symbolic approaches for the knowledge representation, deep parsing for NLP and semantics/discourse models, and machine learning for pattern recognition tasks of any kind.
that's what we (Kyndi) think. Good AI requires ML, NLP, and KRR. ML to acquire the knowledge on which to reason, KRR to represent and reason, and NLP as the glue that holds everything together.
Lets rename "Deep learning" to "Statistical Compression".
Its essentially distilled data from a dataset, limited by size of the neural network. There isn't anything "deep" there, the goal of NN is always towards overfitting the data.
That approach didn't work for "cyber", "quantum" or similar buzzwords. Calling something by its actual meaning isn't easy to sell. Hell, even "hacking" still has the connotation of some criminal activity, although it is widely known that the process of hacking in itself cannot be classified as benign or malign! But it doesn't matter. What matters is what public thinks of it hearing the buzzword. "Deep learning" is the same thing: "I have no idea what it is, but you know what I mean right?"
I have never once encountered this misconception until now. Every explanation of deep learning I’ve seen begins with an explanation of the origin of the phrase.
A few weeks ago someone submitted an entire article to HN that consisted of someone complaining that deep learning wasn't really deep, in the deep & meaningful sense. They clearly misunderstood. Was pretty funny.
Lets rename "Deep learning" to "Statistical Compression"
The previous name for what is now called “machine learning” was predictive statistics. But a better name for deep learning is “machine intuition” since it seems to work pretty well but can’t be readily explained
It works "pretty well"
in well-defined complete-information domains like Go.
The training in that case is actually about improving the next dataset from which higher-quality patterns are extracted.
1.We have a NN that doesn't know how to play and makes random moves.
2.Blank slate dataset filled from random games.
3.NN learns to produce better games by weighting wins vs losses, making losing moves less likely and winning moves more likely.
4.NN produces slightly better dataset.
5.Repeat #3 until you have covered a large chunk of go openings and have pretty good estimation for move quality.
6.NN becomes the condensed statistic pattern data extracted
from billions of games, able to estimate value of a move more accurately than humans.
Still in some rare cases it "condensed pattern database" will make mistakes due gaps in data or wrong correlation that NN hallucinated from its connections.
works "pretty well" in well-defined complete-information domains like Go
But no one can definitively say “my AI credit scoring system didn’t decline his mortgage because he is black” and that’s the explainability bar that needs to be passed.
Right now the state of it is “gut feel” with an unknown amount of unconscious bias.
We are a long, long way from cracking open a neural net cat classifier and pointing at the exact equation for the whisker coefficient or the nose cuteness factor.
This discussion reminds me of NASA's CLIPS [1], particularly FuzzyCLIPS. I know that is basically a rule engine/expert system basis at its core, but FuzzyCLIPS was an interesting variant that try to do inference off of CLIPS rules and facts.
I always wondered if rules and facts can be seen as high-order abstractions that represent the low-level data that is encoded into neural nets. Would that mean that a system of symbolic logic, or rules and facts can in fact accomplish the same thing, just in a different, higher order way?
Then when you get into combining fuzzy with known, it's like different organs in the brain combining to serve different functions. When I know a rule and recall it, I believe it to be true and act immediately on it. When I'm not sure, I get fuzzy and try to deduce it. The more data and inputs I have about a given environmental condition, problem, subject, input, fact, etc., the easier it becomes to "reason" using fuzzy links to try to work out the answer. Unless I get information overload or the complexity exceeds one human brain's capability.
I've always felt that techniques are complementary and probably we employ multiple subsystems in our brains together to act like humans.
One problem with ai (or general ai) is ai lives in a box, it experiences what we tell it to experience, unlike a child who learns organically with what it's exposed to randomly while trying to survive and thrive.
Perhaps the answer is to figure out all the things that make people tick via deep learning and other techniques, build one huge open api/db for vision/audio/movement in 3d planes/ethics/morality/understanding/discernment/language and then a more general ai could simply send out calls to api's of data already trained on specific foundations and create a 'whole' brain from multiple 'sub' brains (pre-learned open knowledge).
Unless we literally learn to clone the human brain technologically, in it's entirety.
I think another issue is not all working in AI are neuroscientists, and it may take understanding every mapped out facet of the human brain to come close to re-creating that in humans, it all can't be deep learning unfortunately, because that's not how we learn and we're the best guess solution to 'intelligence' as the only intelligent race (we know of) in the universe.
It seems there's nothing logical about human cognition, it just ends up resembling logic because movement through space either works or it doesn't, and humans have the ability to assign physical analogies to abstractions and to respond to those metaphors with the same heuristics and physical reflexes that it was born with; heuristics and reflexes designed over a few billion years to allow your ancestors to survive and reproduce.
The problem with training things on artificial datasets is that they are not rich (resolution is too low) or surprising (generated by simple algorithms.) An AI that worked as well as the human brain would just end up reading them back to you. The real world is, information-wise, incompressible. It has such fine distinctions that its impossible to examine them beyond a certain level without the act of observation changing them.
I always thought there was some potential in using robotic bodies with good sensors that fed into some processing theory of mind in order to grow an AI.
I think finding a way to quite literally build a 'brain' that mimics all aspects of the human brain including: Size, shape/areas/etc that is essentially a mechanically cloned brain like a mechanical heart or lung (Similar to the organs in Bicentennial man), may be the only way to get true AGI.
Simply put we need to work more on mirroring nature and biological brains to get computer brains that function like human brains. Even neuroscience still debates on what consciousness even is.
> it all can't be deep learning unfortunately, because that's not how we learn
If you could humor me, why can't it all be deep learning? It seems like a robust model for how we learn: we start with some priors (random weights), experience something that either confirms or negates those priors (sampling), and then adjust our priors based on that experience.
Of course there are a wide variety of architectures and the human brain is more sophisticated than one conv net. Our brains have on the order of 100 billion neurons and I believe the largest neural networks are on the order of 10's of millions. I would argue that a combination of better architectures and scale could simulate human intelligence.
i dont think we start with random priors. We start with a set of priors that are geared for certain things - facial recognition, language etc. The environment then fine tunes them.
There is no way that the genome encodes synaptic layouts to such a detail as to specify an algorithmic prior. At best it specifies, loosely, the generic architecture of how many neurons, how many layers, how tightly folded, and where the inputs (senses) connect. We develop the same algorithms because we all start with the same priors in largely the same gestational and early infant environments.
No, we don't "learn" how to balance, how to identify objects and navigate 3d environments. All of that is highly codified in our DNA from hundreds of millions of years of evolution. Human babies, similar to kangaroos, are just born too early for those systems to have developed.
Part of it could be hardware related, we may need special chips just geared for this, with special coded sensors.
Also, not everything we do is in our brain -- there's also muscle memory, and even tissue inside the gut stores some memories and 'knowledge'. Technically every cell in our body has some purpose/goal and can contribute to our entire being/feeling/mood/etc.
There's a lot we don't know about conscience that makes developing an artificial one, very tough. That's why it's not estimated to happen till after 2050. I don't doubt it will happen, I don't doubt deep learning as is will be crucial to the discovery, I just think there's possibilities that undiscovered and better/alternate techniques might be discovered that could potentially trump dl as we now know it.
There's a lot of things we do not know to get us to general ai, and to get there we may need to jump outside some 'boxes' that ai researches have put themselves into.
Seen a few people claim from time to time that real world experience is the "special sauce" of general AI. I've never seen anything to substantiate this claim. We all live in a box, we all experience our world through simple electrical signals.
Other than being unsubstantiated, the real world only runs at 1x, simulated world can run at any rate your computing platform supports.
Much of our reasoning develops from metaphorical comparisons with our embodied experiences. E.g. it is certainly true that we gain spatial awareness from embodiment, and it is worth noting how much of our language of reasoning uses spatial metaphors. Can we expect a non-embodied AI whose only view of the world is text we feed it to develop a similar understanding?
One feature also is hardware capabilities (storage + processing), the human body/mind is extremely capable. It can store an unknown amount of data, it boggles my 'mind' at the amount that can be stored in such a small area compared to today's hdd's and sdd's, then how fast we can bring up those memories - yet we don't always have those memories in quick access points, sometimes we have to think real hard to remember something. We also don't remember EVERY single detail about something, only key characteristics, where I think a lot of AI tries to learn/store/remember everything about something.
The storing of that can take up precious resources, to create service droids that can do everything using AGI we need the computer that runs it to be essentially the size of a real human brain. Which we need to find ways to map AI tech more closely to biology.
There's also the fact that we are born with instincts built up over millenia, passed down. We also absorb random tidbits of data with no actual purpose. We watch the news and our toddler gets excited at the commercials, I don't know what he's learning from it, but getting him to turn his attention away from it is nigh impossible.
You can see the processing going on, it's amazing to watch him learn. I just think we need to think outside the current ai 'boxes' with dl/ml and consider their may be whole swath's of things we're missing, whole pieces of the puzzle before we get to AGI or the Singularity.
This is a valid question, I don't know why you (originally) got downvoted.
The answer I think, for the applications in industry at least, is that PRISM and other probabilistic programming langauges are not as developed as Prolog, a language that has been around for a good four decades now. Swi-Prolog in particular, is a free and open-source Prolog interpreter with an IDE, a graphical package and a veritable somrgasbord of libraries, including an http server, json and xml processing, encryption, interfaces to SQL and other databases etc etc etc. Swi is in active development, compatible with YAP-Prolog (currently the fastest implementation) very well supported and documented and has an active community.
It's a lot easier to setup a production environment in Prolog, especially using Swi, than in pretty much any other logic programming language, including the probabilistic logic programming ones.
Btw, I noticed that Kyndi list experience with Logtalk as a "plus" for one of their career opportunities (of course I looked :) which means that they're probably using Swi-Prolog.
But its not clear to me that encoding productions in source code adds any value over encoding them in the various other forms used by expert systems of old. Prolog also compels binary constraint satisfaction, recursive descent parsing, backtracking, and depth-first resolution — all undesirable constraints that are easily avoided using other fact representations or resolution engines.
Until the startup in question can make a compelling case for using prolog, much less avoiding the brittleness inherent in expert systems' resolution model, count me a disbeliever that their 'better way' really is.
I wouldn't say that backtracking and depth first search (not resolution) are undesirable. Far as I'm concerned they're pragmatic choices that minimise the amount of resources necessary to perform resolution theorem-proving.
One can encapsulate backtracking simply as an iterator over a stream of answers to a query. The SWI_prolog ecosystem provides a nice "logic engine" abstraction for that, see http://www.swi-prolog.org/pldoc/man?section=engines . That gives something similar to Python's yield operation. Backtracking inside an engine allows one to implement some neat algorithms in space proportional to your data, without invoking garbage collection.
I would hate for my logic engine to be required to backtrack every time a candidate interpretation was evaluated. AFAIK, prolog allows no alternative. Nor can it support the many powerful probabilistic extensions for rule-based reasoning that arose 25(?) years ago, nor the many improvements and variations on impasse resolution.
IMHO, the choice of prolog unnecessarily straitjackets a modern production system, making the engineering approach of this startup that much less powerful, flexible, or viable than it should be.
Also, the (!/0) operator allows you to prune choice-points and control backtracking so that it is not unconditional. Many Prolog implementations support (->/2) operator, as well as ((*->)/2) -- the soft cut, for more fine-grained control. SWI has a nice library for even further control over backtracking as well: http://www.swi-prolog.org/pldoc/man?section=solutionsequence....
Also good Prolog programmers usually are not overly concerned with backtracking, as they usually have strong knowledge of modes and determinism. See: http://www.swi-prolog.org/pldoc/man?section=modes. This is mostly second-nature to a skilled Prolog developer.
These days Prolog is general purpose, powerful, pattern driven programming language that (once properly learned!) can put together database, persistence, networking, NLP crunching in one modular program unit. Associating it with expert-system or rule-based programming is a typical AI-winter misconception. Well, a similar one was also floating about deep-learning during the same frosty season :-)
The New York Times and Washington Post both use a "fingerprint" method to identify unique users using things like your browser version and IP address. I like using Firefox nightly because a quick update to Nightly will make it appear that I'm a new user and will reset my free "monthly" article limit. That's also why switching browsers will work for some people, but not others.
>> If the reach of deep learning is limited, too much money and too many fine
minds may now be devoted to it, said Oren Etzioni, chief executive of the
Allen Institute for Artificial Intelligence. “We run the risk of missing other
important concepts and paths to advancing A.I.,” he said.
I was very pleasantly surprised this year to see the Symbolic Machine Learning
course at Imperial College London well-attended by about 30 students each
session. The course basically teaches the principles and practice of Inductive
Logic Programming, a set of algorithms and techniques that learn logic
programs from relational data [1]. It is an optional course at Master's level
so I was originally expecting to see maybe half a dozen people attending (I
assisted with a couple of the sessions and followed the rest to brush up on my
background).
I discussed my surprise with my supervisor who is one of the tutors on the
course. His opinion was that logic-based AI is much closer to the material
that most Computer Science students are familiar with, than statistical and
probabilistic AI. It's true that, although many CS courses have started to
include statistics and probabilities in their curriculum (the one at Imperial
sure does) the mainstay of computer science are logic-based, discrete maths,
complexity and computation theory and language theory. The jump from there to
GOFAI is just a short hop; in many ways, the two fields are really one subject.
My intuition is that the boom in deep learning is partially being fed by
disciples of fields well outside CS -maths, physics, bio-sciences etc- jumping
on to the, well, bandwagon, motivated by the deep learning frenzy in industry
and not so much by any interest in AI or even computers. At the same time,
there is plenty of traditional CS talent waiting in the wings. It may be that
a revisiting of the good old ways of knowledge representation and inference
may not be completely out of the question [2].
Of course, there's always the chance that the students only came to the course
because it had "machine learning" in its name :)
__________________
[1] More the point, both of the outputs and inputs of ILP are logic programs,
i.e. binary relations. Most of the work uses Prolog but there also exist
techniques to learn Answer Set Progarmming, Constraint Logic Progarmming and
of course Probabilistic Logic Programming programs. In any case, the
homoiconicity of logic programming languages allows a newly learned hypothesis
to be immediately reused as data to learn a new concept, in a way that is
probably impossible with statistical techniques. And of course, learning is
extremly sample efficient and the resulting programs generalise vey well.
I bet you had never heard of any of this before now :)
[2] To be honest, my guess is that the way forward is a combination of the two
approaches- perhaps, some monster combo of deep learning for sensory
perception, logic for inference and probabilities for final decision making.
One way or another, I expect the AI researcher of the future will have to be a
jack of all trades - and master of all of them at once. Learn to juggle.
Deep learning has produced a lot of interesting results over the past few years, but we're still miles away from anything that even approximates human intelligence. When I hear someone like Elon Musk say something like "Autonomous vehicles can drive without Lidar because humans can drive without Lidar" it makes me cringe.
The human brain is unimaginably complex. Even if we created a neural network on the order of the 100 trillion or so connections in the central nervous system we would still not even be close. Each one of those synapse is a complex, constantly adapting chemical computer which is sensitive to the interaction of dozens of neurotransmitters. Gradient descent is to neural adaptation as minecraft is to photorealism.
The idea that artificial intelligence will be achieved by adding a few layers to a "deep" neural network, or by throwing more data at it is laughable.
> Each one of those synapse is a complex, constantly adapting chemical computer which is sensitive to the interaction of dozens of neurotransmitters.
Granted. On the other hand, electric signals can propagate across a chip several orders of magnitude faster than the chemical-electrical signals neurons use to communicate. So while our current path won't lead us to an artificial brain that works exactly like the human brain, it might still make one that's just as smart through pure brute force.
A biologist in 1902 could very reasonably have scoffed at the heavier-than-air flight enthusiasts who pointed at birds as a proof of concept. The biologist could point out that the complex, many-part motion of a bird's flapping wing was far beyond the crude mechanical processes inventors were trying to use to replicate it. And they'd be right. But in a much more important way, they'd be wrong.
I have no idea whether we're on the path to AGI or not, but I do think that "we're not building machines that work exactly like the brain does" is a poor counterargument.
> "we're not building machines that work exactly like the brain does" is a poor counterargument.
It's not that artificial intelligence has to work exactly like human intelligence; my point is that the system that produces human intelligence is many orders of magnitude more complex than what passes for AI these days.
I think there's an idea out there that because deep learning allows computers to solve some problems that we thought were in the exclusive domain of humanity even a few years ago, that the refinement of these technologies and Moore's law will steadily close the gap between AI and human intelligence. I would venture to guess there are a lot of hard, novel problems left to solve before we get anywhere close.
edit:
It's also not the first time we've thought we had intelligence "figured out". In the middle of last century, there was a lot of hubris around the field of neuroscience as we started to understand the brain's topology, and could map specific functions to brain regions. The feeling at the time was that we would refine our map, and sooner or later once we figured out what all the parts did, we would have a total understanding of the human mind.
This paradigm led to some horrifically misguided practices. For instance the use of lobotomy to treat psychiatric disorders: if the brain is just a collection of functional regions, why not just cut out the bad bits right? Turns out it's not that simple.
You can also train a horse to tap it's foot once for "yes" and twice for "no". You can ask it a bunch of questions, and it can seem like you're having a conversation, but it turns out the horse is just paying attention to a few details about your body language and performing the action you gave it carrots for. It might seem like intelligence, even to the trainer, but it's a parlor trick.
I don't mean to diminish the power and value provided by deep learning, but it doesn't change the fact that pattern recognition is not equivalent to intelligence.
Perhaps Elon meant to reason the following: if humans can drive without Lidar, then it is possible to drive without Lidar, then it is possible to build an autonomous car that doesn't use an expensive piece of equipment such as Lidar.
The human brain was not designed, but evolved. Why do you think we can't do better when in fact we have done exactly that repeatedly, focusing both on aspects of human biology and other non-human biological systems.
> The idea that artificial intelligence will be achieved by adding a few layers to a "deep" neural network, or by throwing more data at it is laughable.
It's called "cargo cult science" and it's an important part of the AI hype train to keep funding coming.
The currently trendy "AI" looks more like massive data mining with powerful ML to me, that's very good for certain tasks but brings us nowhere near real AI. The knowledge representation problem has not yet been solved. In order to get even just a convincing simulation of AI, let alone real AI, we need a large common sense knowledge base / computational ontology.
One perhaps promising approach are geometric meaning theories/concept representations that allow for logical combinations. For example, Diederik Aerts works in this area. However, to be honest, I don't have the math skills to evaluate his approach. Generally speaking, logical modelling is too limited for good concept representations - especially classical 90s AI like in default reasoning and other nonmonotonic logics -, whereas traditionally geometric representations suffer from problems with representing logical inference and quantification. IMHO, that's a problem worth looking at. (Admittedly, I'm a bit biased towards symbolic AI like the people in this article.)